Commit Graph

303 Commits

Author SHA1 Message Date
c915d68c97 feat: pagination support for logs request 2023-09-08 01:41:36 +03:00
62ae3ada4a feat: show help message if no command specified 2023-09-05 18:26:22 +03:00
6fcb5449c3 feat: add ability to skip migrations
This feature can be used in order to handle conflicting migrations
2023-09-02 14:30:50 +03:00
a1db4dc8b8 add ability to partition tree before calculationn 2023-08-27 01:55:57 +03:00
f6081507c0 small improvements on code smell
* fix some grammar/typo errors
* change some statements to be more clear
* use pattern matching for enum processing
2023-08-25 04:15:10 +03:00
33e68a59e2 use http client class for all http requests 2023-08-23 03:27:42 +03:00
5dc6df11c5 verbose subprocess exception handle annd quite git 2023-08-21 01:35:43 +03:00
c26a13c562
Remote call trigger support (#105)
* add support of remote task tracking
* add remote call trigger implementation
* docs update
* add cross-service upload
* add notes about user
* add more ability to control upload
* multipart upload with signatures as well as safe file save
* configuration reference update
* rename watcher methods
* erase logs based on current package version

Old implementation has used process id instead, but it leads to log
removal in case of remote process trigger

* add --server flag for setup command
* restore behavior of the httploghandler
2023-08-20 03:44:31 +03:00
ca60317750 add dummy type fields for pkgbuild generators
This field is required in order to pass config validation in case if
section name differs from default one. Also by default keyring_generator
and mirrorlist_generator have been renamed to keyring-generator and
mirrorlist-generator respectively for consistence
2023-08-17 16:02:15 +03:00
1384efb31d close descriptor after uploading archive on github 2023-08-15 02:31:46 +03:00
a1d0e993a8 resoolve dependencies by using local cache too (#107) 2023-08-14 02:31:24 +03:00
572880eb73 add ability to read values from environment variables
It makes sense to read some values from environment. In particular this
feature is useful in case of running application in containers in ci/cd

See #108 for more details
2023-08-14 01:48:08 +03:00
d9eaf17a11 remove unused absolute path validator (#106)
Extracted path is always absolute, so there is no need to check it
2023-08-13 20:48:07 +03:00
95e29d16bb Local packages support improvements (#104)
* handle git author correctly
* make remote source required argument
2023-08-13 15:45:53 +03:00
1f2d56e605 make auth.salt parameter optional
Used implementation of the hasher includes salt itself, thus additional
salt is optional and can be safely (in terms of security) treat as empty
string
2023-08-11 16:31:47 +03:00
1baf04998d full support of pep517
Since llast upgrade build is broken. Lets fully migrate to
pyproject.toml. Note for maintaners: because data_files option is
deprectated (see https://github.com/pypa/setuptools/discussions/2648)
you will have to install files manually inside your packaging process
2023-08-11 03:55:31 +03:00
3a88d00db0 automatically bump pkgrel on version duplicates
The new --(no-)increment flag has been added to add, update and rebuild
subcommands. In case if it is true and package version is the same as in
repository, it will automatically bump pkgrel appending (increasing)
minor part of it (e.g. 1.0.0-1 -> 1.0.0-1.1).

Inn order to implement this, the shadow (e.g. it will not store it in
database) patch for pkgrel will be created
2023-08-08 03:14:47 +03:00
4abe3b8963 remove napoleon contrib dependency 2023-08-08 02:53:08 +03:00
237fec3f85 fix issues with remote pull triggers (see #103)
* The issue appears when repository contains PKGBUILD in root. In this
  case it will copy tree with loosing package information, because
  the repository will be cloned to temporary path with random generated
  name
* The issue appears when branch which is different from master is used
  for any reposittory with git files (e.g. single-pkgbuild repo or repo
  with submodules)
2023-08-08 02:34:44 +03:00
263c53bac5 rase 405 error in case if GET login method is used whereas no aioauth
library installed
2023-07-23 03:01:25 +03:00
6743f1d62a optimize schemas import 2023-07-23 02:24:39 +03:00
b7852f55c8 remove unsafe flag from handlers
This flag became reduntant there and tree creation has been moved to
lock
2023-07-07 03:25:05 +03:00
b80ea80e9d add salt generator to setup command instead 2023-07-06 19:16:49 +03:00
8e9da5baab register dependency package before build
If package has been added as dependency, the service miss remote as well
as causes some 400 errors in reporter
2023-07-06 03:11:19 +03:00
a443abb94e handle packages load from aur by package name also
In general package names array may not contain package base, thus it
leads to inability to load packages from aur by its base during update
process
2023-07-01 15:55:04 +03:00
61c565ab0d explicitly pass user agent for the arch linux sites requests 2023-06-26 02:52:08 +03:00
10100b20e1 print configuration paths in dump command 2023-06-11 15:11:34 +03:00
2922bb9d72 remove salt generation from users handler
It causes issues, because users handler is operating with service user,
but writtinng salt requires root privileges
2023-06-05 05:25:10 +03:00
4b984afb64
packagers support (#100) 2023-06-05 02:37:19 +03:00
041e9536bf use cached property instead of custom __getattr__ implementation 2023-05-31 19:17:16 +03:00
1625fddccd support check dependencies 2023-05-31 19:17:16 +03:00
8f2c33fe5d
mimic parent arguments during spawn process (#99) 2023-05-30 18:09:35 +03:00
d064e46af6 do not terminate child process explicitly
This feature sometimes causes the main process termination. Since the
child process has been already terminated (or going to) there is no need
to terminate it manually
2023-05-29 17:27:04 +03:00
5627c9cca0 fix case where it was impossible to create session if user and password
are required

Issue appears if auth is enabled and unix socket is disabled and caused
by 7f5e541120
2023-05-29 17:23:57 +03:00
8731cee8ea runtime logger handler selector 2023-05-29 03:41:11 +03:00
8f0a6cf6b5 allow to filter packages for rebuild by their statuses 2023-05-25 17:41:54 +03:00
1cf027cb55
Support type triggers (#96)
* implement mirrorlist package generator

* implement keyring package generator

* docs update

* do not skip empty lines

* fill remote source for local packages

* faq update
2023-05-25 16:27:16 +03:00
70e9e3e822 migrate to the latest devtools & archlinux git changes
This migration includes
* removal of community repository as it is no more
* fixed paths for devtools configurations
* migration of archlinux packaging git url
2023-05-22 16:51:27 +03:00
8c55438140 disable debugpanel for now 2023-05-04 14:30:27 +03:00
2ff56965d9
PEP-673 use Self as return type for classmethods (#94)
* PEP-673 use Self as return type for classmethods

* add dummy test file

* remove python3.10 compat
2023-05-04 03:28:08 +03:00
9dc6d56a8d fix flapping tests for oauth
Original implementation sends requests to httpbin which sometimes might
not be available. With proposed changes we are blocking redirects and
just check request itself
2023-05-03 14:53:35 +03:00
31f47b8069 fix some pycharm warnings 2023-04-17 02:23:50 +03:00
e42ca95789 suppress status errors also if option is set 2023-04-15 05:05:55 +03:00
abf29083c4
PEP-585 complaint: remove type aliases (#93) 2023-04-12 04:09:28 +03:00
9503a9f2ae try to remove unknown packages from api 2023-04-06 18:14:36 +03:00
58379e7bf3 optimize imports 2023-04-06 00:24:39 +03:00
e08ab2db10 extract schemas automatically from views 2023-04-06 00:24:39 +03:00
8f4a2547e8 use api generated docs instead of comments (#92) 2023-04-06 00:24:39 +03:00
7f5e541120 execute request in context methods instead of handling them each time
manually
2023-03-23 12:43:04 +02:00
df23be9269 gracefully terminate web server
In previous revisions server was terminated by itself, thus no lock or
socket was removed. In new version, graceful termination of the queue
has been added as well as server now handles singals
2023-03-06 01:13:41 +02:00
a274f91677 simplify login ttl processing 2023-02-24 16:52:55 +02:00
13faf66bdb add more validation rules 2023-02-23 15:18:56 +02:00
4fb9335df9 add ability to read cookie secret from config 2023-02-22 18:47:56 +02:00
37e57c13c8
update dependencies before build (#91)
Old implementation has used add step in order to fetch dependencies,
which could lead to build errors in case if dependency list was updated.

New solution uses dependencies which are declared at current version and
fetch them (if required and if enabled) before update process.

Closes #90
2023-02-12 06:02:30 +03:00
19bb19e9f5 handle .gitignore file correctly in remote push trigger 2023-02-11 04:41:24 +02:00
3a4e8f4d97 mask mypy warning
The newest mypy produces the following warning:

src/ahriman/application/handlers/search.py:43: error: Non-overlapping identity check (left operand type: "Union[_DefaultFactory[Any], Literal[_MISSING_TYPE.MISSING]]", right operand type: "Type[List[Any]]")  [comparison-overlap]

which is more likely caused by updated dataclass models to protoocol (however decorators are still calllable). This commit masks problematic line from checking
2023-02-09 22:46:08 +02:00
4db8ad8e8d hide passwords and secrets from repo-config subcommand by default 2023-02-05 16:44:48 +02:00
917ec48be5 handle architecture specific fields for dependencies
This change requires srcinfo at least 0.1.2 version. Unfortunatelly aur
api don't support architecture specific arrays for now, so we just leave
it as is

Closes #82
2023-01-31 14:34:09 +02:00
0a2ba4ae07 filter empty packages from database
In some cases (e.g. during addition of the package to build queue) we don't have
full information about package inself; in these cases we produce lines
with empty architecture, which duplicates normal ones.

This commit changes architecture column type to required and also
filters packages which don't have architecture set yet.

Closes #83
2023-01-30 17:57:13 +02:00
9d7f63e549 trim version from provides list
Closes #87
2023-01-30 17:27:34 +02:00
25eee9ca5e add ability to suppress http logging errors (#86) 2023-01-30 17:19:01 +02:00
5af84955ac
calculate dependencies based on package information (#89) 2023-01-30 17:28:05 +03:00
17466d8d37 make oauth client trully optional (#84)
Same old song, after migraiton to packages some optional modules are
being imported globally which lead to making hard dependency
2023-01-25 15:25:42 +02:00
9e4e3b701b enable lock for web service 2023-01-18 01:39:55 +02:00
616a1950ae add ability to override pacman mirror in devtools configuration
This commit also extends configuration of the multilib option, adding
the ability to exlcude multilib repository from repositories list

Note, that in order to support repository list and mirror correctly,
alpm configuration section is now architectture specific
2023-01-15 18:05:15 +02:00
d7356926c4 consider vcs flag and vcs_allowed_age during local packages update 2023-01-12 15:37:05 +02:00
43a7d09cab add ability to check for service updates 2023-01-12 15:37:05 +02:00
77954b988b subcommand review
Some commands have been moved to another group and thus having another
default name (old subcommands are still available...for now):

* daemon -> repo-daemon
* key-import -> service-key-import
* repo-clean -> service-clean
* repo-config -> service-config
* repo-config-validate -> service-config-validate
* repo-setup -> service-setup
* repo-shell -> service-shell
* version -> help-version
2023-01-12 15:37:05 +02:00
0239fb50b6 expose trigger configuration schema
Note that this commit contains the following breaking changes:

* remote pull and remote push triggers are now enabled by default (with
  empty target list)
* remote pull and remote push triggers now require target option to be
  set (old behaviour had fallback on `gitremote`)
* validation is now considered to be stable, so it is enabled by default
  in docker image (can be disabled however)
2023-01-10 03:33:52 +02:00
d942a70272
add config validator subcommand (#80)
* add config validator subcommand

* add --exit-code flag

* docs & faq update
2023-01-09 18:22:29 +03:00
014007ade3 review unsafe commands access
Some commands were made unsafe in old versions, but nowadays they can be
run without having special privileges.

There was also a bug in which status commands were not available if you
are not ahriman user and unix socket is used. It has been fixed by
switching to manual socket creation (see also
https://github.com/aio-libs/aiohttp/issues/4155)
2023-01-04 17:45:44 +02:00
04e5a263b7 add notes about documentation and methods inside class
Because I always forget which way I used before
2023-01-03 01:53:10 +02:00
caca1576c8 Correct way to allow setting context with existing
This reverts commit 5c4d3eeffd.

Original solution has introduced special workaround (strict flag) which
contradicts the concept of immutable context. Moreover, it introduces
possible side-effects, because child process will use the one set by
parent instead of having own one.

The correct solution is to re-create context in process entry point

Sorry, it was Jan 1 and I was drunk :(
2023-01-03 00:48:14 +02:00
5c4d3eeffd allow setting context with existing
In case of running command from web interface, it will raise exception
because context has been copied with subprocesses
2023-01-02 03:21:15 +02:00
2c2eae2334 remote all gitfiles in git remote trigger
In case if there is .gitignore file with asterics, the pkgbuild upload
would not appear
2023-01-02 01:45:50 +02:00
f984ea75d0 fully lazy handle load
In case of immediate handle load it would try to sync databases (or at
least to create database files), which is not possible in case if
command is run as non-ahriman user. This commit makes handle load lazy
and allows to run some commands as non-ahriman user
2022-12-31 14:48:21 +02:00
81d9526054 improve VCS packages checks
* Unlike older version, currently service will always try to pull AUR
  package to check version. Previously if no-vcs flag is set, it would
  ignore VCS packages completelly
* Introduce build.vcs_allowed_age option. If set, it will skip version
  calculation if package age (now - build_date) is less than this value
2022-12-30 21:19:23 +02:00
7de94a8daa add repo-structure subcommand
This commit also changes Tree class, replacing load method by resolve
2022-12-27 10:35:03 +02:00
8864855c14
write patches via gitremote push trigger (#79)
* write patches via gitremote push trigger

* implement context variables intead of custom database class
2022-12-25 02:10:38 +03:00
7df4adfc9d replace alert bodals with toasts 2022-12-09 11:33:19 +02:00
262462d3c3 improve wording in documentation 2022-12-02 15:45:01 +02:00
7aa91f9e2e do not trigger update on sign command 2022-12-02 01:41:23 +02:00
01eda513cf improve setup command by --makeflags-jobs argument and fix repository sign on creation 2022-12-02 01:41:23 +02:00
0161617e36 implement support of unix socket for server
This feature can be used for unauthorized access to apis - e.g. for
reporting service if it is run on the same machine. Since now it becomes
recommended way for the interprocess communication, thus some options
(e.g. creating user with as-service flag) are no longer available now
2022-12-02 01:41:23 +02:00
4811dec759 do not invoke configuration write in case if no salt or user was written 2022-12-02 01:41:23 +02:00
09623c20d5 add rebuild implementation to interface 2022-12-02 01:41:23 +02:00
5073c80af1 add key-import button to interface 2022-12-02 01:41:23 +02:00
d36e851a29 render httpunauthorized as html in plain http requests 2022-11-22 23:40:27 +02:00
3bca780bdd check log record in handler instead of client 2022-11-22 02:30:37 +02:00
78e6b48c24 ask users to repeat password
In case if password is asked via getpass, it is possible to make typo
and user will not see the mistake. In order to avoid it, additional
confirmation has been added
2022-11-22 02:19:37 +02:00
137d62e2f8
Extended package status page (#76)
* implement log storage at backend
* handle process id during removal. During one process we can write logs from different packages in different times (e.g. check and update later) and we would like to store all logs belong to the same process
* set package context in main functions
* implement logs support in interface
* filter out logs posting http logs
* add timestamp to log records
* hide getting logs under reporter permission

List of breaking changes:

* `ahriman.core.lazy_logging.LazyLogging` has been renamed to `ahriman.core.log.LazyLogging`
* `ahriman.core.configuration.Configuration.from_path` does not have `quiet` attribute now
* `ahriman.core.configuration.Configuration` class does not have `load_logging` method now
* `ahriman.core.status.client.Client.load` requires `report` argument now
2022-11-22 02:58:22 +03:00
cdd66ee780 fix case when no files were commited in remote push trigger
The issue appears together with --intent-to-add flag for adding new
files. Original testing has been performed by having already added new
files, thus it passed all checks.

This commit also adds `commit_author` option which will allow to
overwrite the author.
2022-11-14 00:59:43 +02:00
551ee670bf rollback cwd parameter for clone 2022-11-11 21:19:27 +02:00
4f21eb6fe6 Fix issue when there is no cached source directory yet (closes #75) 2022-11-11 17:23:15 +02:00
0cd07afa0f use intersection of from_database and depends_on filters for the rebuild subcommand
Old logic used OR condition, i.e. if set from-database, it would ignore
the --depends-on flag. In new logic it calculates dependencies based on
the package list, which can be retrieved from database
2022-11-11 16:02:19 +02:00
9556779ee2 add exec to shell subcommand 2022-11-10 18:52:55 +02:00
8b32bd43db make exception optional argument for check_output method 2022-11-10 18:05:32 +02:00
0eadef597a
replace several store_true keys to booleanoptionalaction alternative (#74) 2022-11-10 18:34:01 +03:00
e58ccdc8ad write .makepkg.conf to home dir instead of repository root (#72) 2022-11-08 17:02:13 +02:00
2a07356d24
add ability to use ahriman pacman database instead of system one (#71)
By default this feature is enabled. On the first run it will copy (if
exists) databases from filesystem to local cache (one per each
architecture). Later it will use this cache for all alpm operations. In
order to update this cache, some commands (mainly package building)
provide `-y`/`--refresh` option which has same semantics as pacman -Sy
does.

Note however that due to extending `Pacman` class some methods were
renamed in order to be more descriptive:
* `Pacman.all_packages` -> `Pacman.packages`
* `Pacman.get` -> `Pacman.package_get`

This commit also adds multilib repository to the default docker image
which was missed.
2022-11-08 17:26:51 +03:00