Compare commits

..

579 Commits

Author SHA1 Message Date
arcanis bb4a0d75fc Release 2.13.8 2024-05-12 11:53:19 +03:00
arcanis bca0df41d1 fix: drop integrity check for javascript
It has been added to improve security, however, it changes over time for
no reason ¯\_(ツ)_/¯ I guess either cdn was hacked or fuck js
2024-05-12 11:49:12 +03:00
arcanis 07b77be6b8 Release 2.13.7 2024-05-09 13:26:40 +03:00
arcanis 2b33510ada fix: parse array variable from command 2024-05-09 13:21:42 +03:00
arcanis 6d05389639 Release 2.13.6 2024-05-05 21:59:30 +03:00
arcanis daf9841717 fix: update integrity checksums for momentjs and daterangepicker 2024-05-05 21:17:30 +03:00
arcanis 0d243a781a refactor: update code to the latest python (3.12+) 2024-05-05 21:17:30 +03:00
arcanis cf2e66a934 fix: remove debug packages together with normal ones (#124) 2024-05-05 21:17:30 +03:00
arcanis f01f35238d Release 2.13.5 2024-04-04 13:33:03 +03:00
arcanis d30d512eb6 fix: update Repo.init to the latest pacman release 2024-04-04 13:16:05 +03:00
arcanis 0437f90e5a build: install base-devel package 2024-04-04 13:16:03 +03:00
arcanis 3cab65855a fix: lazy web component initialization
In some cases (probably slow internet) in place initialization can cause
exception, because elements are not available yet. This commit moves
events initialization to $()
2024-04-04 13:14:17 +03:00
arcanis ecfb615f97 feat: add ability to disable debug packages distribution
The feature is implemented as supplying !debug option to makepkg when
generating package list. In this case debug packages still will be
built, however, they will not be added to the repository
2024-04-04 13:14:17 +03:00
arcanis 243983ee64 docs: update docs 2024-02-10 03:12:09 +02:00
arcanis 812c03d1eb Release 2.13.4 2024-02-09 17:47:01 +02:00
arcanis 01597c531b fix: return only built packages from task
Since the last updates makepkg --packagelist also adds debug packages
which causes errors
2024-02-09 17:37:50 +02:00
arcanis 4fec42eac8 refactor: rename packages http methods to own package
docs: update docs import
2024-01-22 02:20:11 +02:00
arcanis 7574b8e5ce Release 2.13.3 2024-01-13 01:24:30 +02:00
arcanis 0f2e7f45da fix: replace logo and name in title to just icon 2024-01-12 01:25:46 +02:00
arcanis 5956a8720b Release 2.13.2 2024-01-08 22:48:55 +02:00
arcanis 8dd4ced5e9 fix: report only unique result entries
since builder intro the triggers are called with merged result, thus it
would lead to duplicated callouts
2024-01-08 22:46:42 +02:00
arcanis 6361c41f76 Release 2.13.1 2024-01-08 21:17:35 +02:00
arcanis 270084bb39 fix: do not raise 404 in case of unknown package on patches endpoints
Previous improvements raise 404 error in case if no packages were found
for patches endpoints. However, in case of multirepo setup this feature
doesn't work properly because package can be located in any other
repository different from default
2024-01-08 14:32:40 +02:00
arcanis f89a5252de build: pass ssh agent to tox release env 2024-01-08 14:22:53 +02:00
arcanis 8cafdb52e5 Release 2.13.0 2024-01-05 22:48:03 +02:00
arcanis 203ebad817 ci: explicit isolated build for old ubuntu tox 2024-01-05 22:47:28 +02:00
arcanis 9f471d11a7 docs: add comments to configuration 2024-01-05 22:24:37 +02:00
arcanis 2ea8a4a07f test: add pytlint imports plugin and fix errors 2024-01-05 19:52:51 +02:00
arcanis 856bbc30d4 refactor: fix pylint warnings in tests 2024-01-05 16:40:38 +02:00
arcanis c88f97c36e refactor: simplify lock processing in worker trigger 2024-01-05 16:11:32 +02:00
arcanis 174d7578a0 refactor: split Path elements to / and first directory 2024-01-05 15:22:46 +02:00
arcanis 8b26035fc9 docs: url spelling 2024-01-05 12:37:23 +02:00
arcanis fc9391dd5f chore: add index page recipe 2024-01-05 12:34:55 +02:00
arcanis ed7ed5d5d9 feat: read username if email is not available for oauth provider
Also add recipe for OAuth with GitHub setup
2024-01-05 12:28:31 +02:00
arcanis 9e1cf3cde7 refactor: use root RequestException instead of the one from subpackage 2024-01-05 11:48:38 +02:00
arcanis 08086e8ba8 fix: safe urls for packages
String catenation used for url generators didn't encode package names
which could lead to missing data in case if e.g. there is slash (/) in
package name
2024-01-03 14:28:20 +02:00
arcanis 1a61da7ab9 refactor: remove custom access logger
It is fine when application is able to log request, however, normally it
produces a lot noise, which has been handled by adding special logger.
However, nowadays it requires a lot endpoints to be filtered and doesn't
provide any choice.

Instead of it lets disable access logger by default and let users decide
do they need or not to see access log messages
2024-01-03 12:18:50 +02:00
arcanis f4dcbbde18 chore: copyright update 2024-01-03 12:09:10 +02:00
arcanis 7aba67186f feat: raise 404 in case if package is unknown for logs and patches 2024-01-03 12:09:10 +02:00
arcanis 8635ee8953 feat: threadsafe services
In the most cases it was enough to just add lock. In case of worker
trigger, since there is atomic operation on timer, it was also required
to add queue (coz python doesn't have atomics)
2024-01-03 12:09:10 +02:00
arcanis 2d42424477 feat: add workers autodicsovery feature (#121)
* add workers autodicsovery feature

* suppress erros while retrieving worker list

* update recipes

* fix tests and update docs

* filter health checks

* ping based workers
2024-01-03 02:25:24 +02:00
arcanis fdf7a36271 chore: make yamll style consistent 2023-12-29 03:57:13 +02:00
arcanis e9116741a5 refactor: fix warnings and typos, cleanup code 2023-12-28 16:53:45 +02:00
arcanis 1a2327cefd build: install aiohttp-cors from aur 2023-12-28 16:05:48 +02:00
arcanis 29c8c4bfd7 chore: add daemon unit 2023-12-28 15:24:14 +02:00
arcanis e40b773029 test: add tests to check args list
Some parsers are shared between different subcommands. It causes errors
when wwe add new arguments to one of them. This commit adds some tests
to cover those cases (except for well-known differencies)
2023-12-27 15:01:07 +02:00
arcanis ee3ccf70ac refactor: use AppKey's instead of string identifiers for web application 2023-12-27 13:53:23 +02:00
arcanis e4b22fd620 feat: allow to run daemon mode with split packages check (#120) 2023-12-27 03:05:44 +02:00
arcanis 48344f759d feat: add healh-check like endpoint, support of healthcheck in docker compose 2023-12-21 17:49:35 +02:00
arcanis 290e6e7670 docs: add reference how to copy package 2023-12-21 02:14:27 +02:00
arcanis e0e4da6669 fix: drop debug toolbar 2023-12-20 16:26:14 +02:00
arcanis 0e2ad3325c docs: docs review 2023-12-20 10:00:12 +02:00
arcanis 28589cb122 chore: add configuration recipes 2023-12-20 10:00:12 +02:00
arcanis 93793b6424 build: drop makefile
It has been a while since we moved to tox. Lets also drop makefile since
it seems to be useless with tox
2023-12-17 02:26:52 +02:00
arcanis b3ce545517 docs: restore docs for the view 2023-12-15 16:17:19 +02:00
arcanis e51d91740d feat: add ability to disable specific routes (#119) 2023-12-15 14:34:03 +02:00
arcanis 5ddc08fce7 feat: add ability to run build process to remote instances (#118) 2023-12-13 15:38:51 +02:00
arcanis f2f6f6df70 fix: correct url for update requests in remote-call trigger 2023-12-11 15:43:28 +02:00
arcanis 2760b36977 feat: changes screen implementation (#117)
Add support of changes generation. Changes will be generated (unless explicitly asked not to) automatically during check process (i.e. `repo-update --dry-run` and aliases) and uploaded to the remote server. Changes can be reviewed either by web interface or by special subcommands.

Changes will be automatically cleared during next successful build
2023-11-30 14:56:41 +02:00
arcanis a689448854 fix: use event instead of chained timer for daemon
Old solution causes amount of thread to be growing as well as stack is
increased during each iteration. Instead of cycle-free implementation,
this commit just uses while cycle
2023-11-30 13:40:59 +02:00
arcanis aef3cb95bc type: update to the typed aiohttp release 2023-11-23 15:35:38 +02:00
arcanis d72677aa29 feat: forbid form data in html
It has been a while since all pages have moved to json instead of form
data, except for login page. This commit changes login to json data
instead of form one
2023-11-16 16:42:27 +02:00
arcanis 82d1be52a8 feat: add separated web client for ahriman web services 2023-11-14 16:41:33 +02:00
arcanis 7536d6bb82 docs: move timeout settings from web to status group 2023-11-13 16:53:16 +02:00
arcanis b050c409cf Release 2.12.2 2023-11-13 12:26:00 +02:00
arcanis d77cf7c4bb style: highligh __del__ methods in formatting 2023-11-13 12:16:30 +02:00
arcanis e03fcbfab5 fix: fix some security issues 2023-11-12 22:22:05 +02:00
arcanis 62dd77317d feat: add separated switch for status reporting 2023-11-11 15:18:17 +02:00
arcanis 95056cfbe7 feat: extend result class 2023-11-10 17:09:01 +02:00
arcanis 2d31a415ce fix: pass packagers object insteaed of string during rebuild 2023-11-08 16:05:27 +02:00
arcanis 5cbeec40f8 build: rename push action to release 2023-11-06 23:25:29 +02:00
arcanis 4a644fc80f Release 2.12.1 2023-11-06 16:31:56 +02:00
arcanis d1aeeda975 fix: show unique upstream urls only 2023-11-06 16:29:18 +02:00
arcanis 53aedbb9aa Release 2.12.0 2023-11-06 15:50:11 +02:00
arcanis c941226a9b feat: add upstream and aur urls to package info modal 2023-11-06 15:46:32 +02:00
arcanis 69cbbfce65 feat: serve favicon in root 2023-11-06 15:46:32 +02:00
arcanis 52d69fa9f6 build: remove spaces from names 2023-11-06 14:04:15 +02:00
arcanis 279ac785a1 docs: correct crossreference in docs 2023-11-06 14:04:15 +02:00
arcanis f51b8e2358 refactor: replace enum with intenum and strenum 2023-11-05 13:26:11 +02:00
arcanis b116e6fa07 feat: allow to pass repository identifier to all database methods 2023-11-05 13:26:11 +02:00
arcanis 7b667c8157 build: remove unused files from tarball 2023-11-05 13:26:11 +02:00
arcanis c51463d36b chore: add metthod definition order plugin to pylint
Also reorder some methods to fix errors
2023-11-05 13:26:11 +02:00
arcanis 79947af013 feat: read old web sections 2023-11-04 13:34:53 +02:00
arcanis e6d6cb006b feat: improve small screens views
also fix case when invalid repository id has been sent
2023-11-01 16:38:53 +02:00
arcanis f553cc059a fix: reset logs window manually 2023-10-31 12:33:57 +02:00
arcanis c99d4f6f0f fix: correct month serialization 2023-10-31 10:13:25 +02:00
arcanis 554827cc57 feat: add patch controls to web, review web, enrich info tab (#115)
* add ability to specify one-time patch on package addition

* support vars in interface
2023-10-29 23:41:20 +02:00
arcanis 8524f1eb20 refactor: drop _check_output class attribute 2023-10-23 02:24:53 +03:00
arcanis 4b6feb9ae6 refactor: add ability to pass anything as environment variable to build task 2023-10-23 02:24:41 +03:00
arcanis ad901067d8 fix: make toggle button works on small displays 2023-10-21 15:18:09 +03:00
arcanis 4c20897e14 fix: write new line for telegram index 2023-10-21 03:56:48 +03:00
arcanis f8cbd07259 fix: do not automatically add unknown local packages
Instead of automatic package addition now it is required to add package
manually after clone. Less magic, plus would allow to use caches for
multi-repo setup (see #109)
2023-10-21 01:33:12 +03:00
arcanis 16ad96d8c6 feat: add ability to run multiple commands on success 2023-10-19 04:34:49 +03:00
arcanis 6bd1636bfa feat: allow to use single web instance for all repositories (#114)
* Allow to use single web instance for any repository

* some improvements

* drop includes from user home directory, introduce new variables to docker

The old solution didn't actually work as expected, because devtools
configuration belongs to filesystem (as well as sudo one), so it was
still required to run setup command.

In order to handle additional repositories, the POSTSETUP and PRESETUP
commands variables have been introduced. FAQ has been updated as well

* raise 404 in case if repository is unknown
2023-10-17 03:58:50 +03:00
arcanis 4eb187aead chore: update comments to the latest mypy release 2023-10-12 20:31:00 +03:00
arcanis f2ac27ccb7 feat: add ability to override oauth2 icon 2023-10-07 18:12:17 +03:00
arcanis b198778810 fix: whitelist environment variables instead of passing all of them
Earlier applied fix d5f4fc9b86 introduced
errors with interaction, because (in docker container) HOME variable was
passed also to subprocesses. This fix limits variables to be passed to
the whitelisted ones
2023-10-02 16:37:20 +03:00
arcanis 57c8ed3557 feat: add __call__ method to printers 2023-10-01 15:14:18 +03:00
arcanis 1859d14f78 feat: load http views dynamically (#113) 2023-09-30 01:24:04 +03:00
arcanis d5f4fc9b86 fix: processes in docker images and pass full environment to
subprocesses

Since 1.0.4 release devtools require systemd to be run in order to
create slice for unit
2023-09-29 03:57:26 +03:00
arcanis 462096ee8a fix: fix documentation building 2023-09-28 15:09:33 +03:00
arcanis 98e594df90 feat: add pagination to packages list 2023-09-11 18:08:26 +03:00
arcanis 657bcdcc0b docs: update documentation to new api calls 2023-09-11 15:57:00 +03:00
arcanis edcc464658 fix: add note about pacman configuration 2023-09-10 23:48:03 +03:00
arcanis 018d9589e1 feat: improve template processing (#112)
* Improve template processing

* docs update, config validation rules update
2023-09-08 23:38:07 +03:00
arcanis a56fe28003 perf: limit clone and fetch by the latest commit 2023-09-08 15:52:59 +03:00
arcanis 03c183d66c fix: correct version check in postinstall script 2023-09-08 14:48:31 +03:00
arcanis efde0b2e86 feat: allow to use one application for multiple repositories (#111)
* allow to use one application for multiple repositories

* update tests

* handle None append argument everywhere

* rewrite repository definition logic

* drop optional flags from docs

* support of new schema in systemd units

* add migration docs and ability to migrate tree automatically

* use repostory id instead

* verbose multiarchitectureerror

* object path support for s3 sync

* fix tests after rebase
2023-09-08 14:01:07 +03:00
arcanis c915d68c97 feat: pagination support for logs request 2023-09-08 01:41:36 +03:00
arcanis 853185c659 fix: mark failed packages as failed in console handler 2023-09-06 18:22:05 +03:00
arcanis 28afcde3fa docs: mock cryptography import due to build errors 2023-09-05 18:37:46 +03:00
arcanis 62ae3ada4a feat: show help message if no command specified 2023-09-05 18:26:22 +03:00
arcanis 6fcb5449c3 feat: add ability to skip migrations
This feature can be used in order to handle conflicting migrations
2023-09-02 14:30:50 +03:00
arcanis 74209acc21 ci: publish docker image to ghcr 2023-08-28 03:21:25 +03:00
arcanis 969352d842 Release 2.11.0 2023-08-27 02:12:21 +03:00
arcanis a1db4dc8b8 add ability to partition tree before calculationn 2023-08-27 01:55:57 +03:00
arcanis f6081507c0 small improvements on code smell
* fix some grammar/typo errors
* change some statements to be more clear
* use pattern matching for enum processing
2023-08-25 04:15:10 +03:00
arcanis 477c473187 force rtd theme
Since Aug, 22 rtd doesn't force its theme anymore, leaving it as default
2023-08-23 15:27:51 +03:00
arcanis 33e68a59e2 use http client class for all http requests 2023-08-23 03:27:42 +03:00
arcanis 6dfe1b92f2 bump pylintrc 2023-08-21 02:55:08 +03:00
arcanis 5dc6df11c5 verbose subprocess exception handle annd quite git 2023-08-21 01:35:43 +03:00
arcanis d3f6ca24c8 review exception raise
In some cases for better readability of logs, exceptions are now raised
without parent exception stacktrace. Also updated docs and contributing
guidelines
2023-08-20 17:03:46 +03:00
arcanis c26a13c562 Remote call trigger support (#105)
* add support of remote task tracking
* add remote call trigger implementation
* docs update
* add cross-service upload
* add notes about user
* add more ability to control upload
* multipart upload with signatures as well as safe file save
* configuration reference update
* rename watcher methods
* erase logs based on current package version

Old implementation has used process id instead, but it leads to log
removal in case of remote process trigger

* add --server flag for setup command
* restore behavior of the httploghandler
2023-08-20 03:44:31 +03:00
arcanis 9ea3a911f7 update gh actions install script 2023-08-17 16:33:02 +03:00
arcanis ca60317750 add dummy type fields for pkgbuild generators
This field is required in order to pass config validation in case if
section name differs from default one. Also by default keyring_generator
and mirrorlist_generator have been renamed to keyring-generator and
mirrorlist-generator respectively for consistence
2023-08-17 16:02:15 +03:00
arcanis 1384efb31d close descriptor after uploading archive on github 2023-08-15 02:31:46 +03:00
arcanis 8c6486c233 contributing guide update 2023-08-14 02:51:14 +03:00
arcanis a1d0e993a8 resoolve dependencies by using local cache too (#107) 2023-08-14 02:31:24 +03:00
arcanis 572880eb73 add ability to read values from environment variables
It makes sense to read some values from environment. In particular this
feature is useful in case of running application in containers in ci/cd

See #108 for more details
2023-08-14 01:48:08 +03:00
arcanis d9eaf17a11 remove unused absolute path validator (#106)
Extracted path is always absolute, so there is no need to check it
2023-08-13 20:48:07 +03:00
arcanis 95e29d16bb Local packages support improvements (#104)
* handle git author correctly
* make remote source required argument
2023-08-13 15:45:53 +03:00
arcanis 1f2d56e605 make auth.salt parameter optional
Used implementation of the hasher includes salt itself, thus additional
salt is optional and can be safely (in terms of security) treat as empty
string
2023-08-11 16:31:47 +03:00
arcanis 1baf04998d full support of pep517
Since llast upgrade build is broken. Lets fully migrate to
pyproject.toml. Note for maintaners: because data_files option is
deprectated (see https://github.com/pypa/setuptools/discussions/2648)
you will have to install files manually inside your packaging process
2023-08-11 03:55:31 +03:00
arcanis 3a88d00db0 automatically bump pkgrel on version duplicates
The new --(no-)increment flag has been added to add, update and rebuild
subcommands. In case if it is true and package version is the same as in
repository, it will automatically bump pkgrel appending (increasing)
minor part of it (e.g. 1.0.0-1 -> 1.0.0-1.1).

Inn order to implement this, the shadow (e.g. it will not store it in
database) patch for pkgrel will be created
2023-08-08 03:14:47 +03:00
arcanis b58d8d96ff Release 2.10.2 2023-08-08 02:54:23 +03:00
arcanis 4abe3b8963 remove napoleon contrib dependency 2023-08-08 02:53:08 +03:00
arcanis 237fec3f85 fix issues with remote pull triggers (see #103)
* The issue appears when repository contains PKGBUILD in root. In this
  case it will copy tree with loosing package information, because
  the repository will be cloned to temporary path with random generated
  name
* The issue appears when branch which is different from master is used
  for any reposittory with git files (e.g. single-pkgbuild repo or repo
  with submodules)
2023-08-08 02:34:44 +03:00
arcanis bd0f850d25 Release 2.10.1 2023-07-28 03:21:19 +03:00
arcanis 5904727da2 do not create application in shell handler
The main reason for having shell handler is to be able to fix if
something (e.g. migrations) goes wrong. In this way we need to reduce
actions inside this wrapper
2023-07-28 03:06:28 +03:00
arcanis 263c53bac5 rase 405 error in case if GET login method is used whereas no aioauth
library installed
2023-07-23 03:01:25 +03:00
arcanis 6743f1d62a optimize schemas import 2023-07-23 02:24:39 +03:00
arcanis 931ff9bd98 Release 2.10.0 2023-07-22 05:28:57 +03:00
arcanis 6b3fc3a6a0 add support of table filter controls (#101) 2023-07-21 02:10:26 +03:00
arcanis b7852f55c8 remove unsafe flag from handlers
This flag became reduntant there and tree creation has been moved to
lock
2023-07-07 03:25:05 +03:00
arcanis 721b447767 fix code block in docs 2023-07-06 19:17:11 +03:00
arcanis b80ea80e9d add salt generator to setup command instead 2023-07-06 19:16:49 +03:00
arcanis 8e9da5baab register dependency package before build
If package has been added as dependency, the service miss remote as well
as causes some 400 errors in reporter
2023-07-06 03:11:19 +03:00
arcanis a443abb94e handle packages load from aur by package name also
In general package names array may not contain package base, thus it
leads to inability to load packages from aur by its base during update
process
2023-07-01 15:55:04 +03:00
arcanis 61c565ab0d explicitly pass user agent for the arch linux sites requests 2023-06-26 02:52:08 +03:00
arcanis 10100b20e1 print configuration paths in dump command 2023-06-11 15:11:34 +03:00
arcanis 2922bb9d72 remove salt generation from users handler
It causes issues, because users handler is operating with service user,
but writtinng salt requires root privileges
2023-06-05 05:25:10 +03:00
arcanis 17f5f41e36 complitely disable signature check for local repository in devtools
It appears that with optional level pacman still tries to validate the
key, which can lead to errors whille processing in docker container
2023-06-05 04:41:03 +03:00
arcanis c22ddd71d9 packager documentation update 2023-06-05 03:40:02 +03:00
arcanis 4b984afb64 packagers support (#100) 2023-06-05 02:37:19 +03:00
arcanis d495163fdd use autodoc_mock_imports instead of custom mocking for docs 2023-05-31 19:17:16 +03:00
arcanis 041e9536bf use cached property instead of custom __getattr__ implementation 2023-05-31 19:17:16 +03:00
arcanis 1625fddccd support check dependencies 2023-05-31 19:17:16 +03:00
arcanis 8f2c33fe5d mimic parent arguments during spawn process (#99) 2023-05-30 18:09:35 +03:00
arcanis d064e46af6 do not terminate child process explicitly
This feature sometimes causes the main process termination. Since the
child process has been already terminated (or going to) there is no need
to terminate it manually
2023-05-29 17:27:04 +03:00
arcanis 5627c9cca0 fix case where it was impossible to create session if user and password
are required

Issue appears if auth is enabled and unix socket is disabled and caused
by 7f5e541120
2023-05-29 17:23:57 +03:00
arcanis 8731cee8ea runtime logger handler selector 2023-05-29 03:41:11 +03:00
arcanis 01a4d68ce6 add repod reference to faq 2023-05-26 12:58:25 +03:00
arcanis 8f0a6cf6b5 allow to filter packages for rebuild by their statuses 2023-05-25 17:41:54 +03:00
arcanis d96e3c97db fix documentation error 2023-05-25 16:31:47 +03:00
arcanis 1cf027cb55 Support type triggers (#96)
* implement mirrorlist package generator

* implement keyring package generator

* docs update

* do not skip empty lines

* fill remote source for local packages

* faq update
2023-05-25 16:27:16 +03:00
arcanis 8e0e57e193 switch to explicit journald logger instead of syslog 2023-05-25 04:44:38 +03:00
arcanis f855e645cc update optional dependencies list 2023-05-24 15:39:36 +03:00
arcanis 8ab0df2bed bump rtd image 2023-05-24 03:58:14 +03:00
arcanis 840921f585 update packaging files
also remove makedependencies after installation in docker
2023-05-22 18:23:06 +03:00
arcanis 6f566b0c6c Release 2.9.0 2023-05-22 17:04:32 +03:00
arcanis d06399db2c update models index 2023-05-22 17:02:55 +03:00
arcanis 70e9e3e822 migrate to the latest devtools & archlinux git changes
This migration includes
* removal of community repository as it is no more
* fixed paths for devtools configurations
* migration of archlinux packaging git url
2023-05-22 16:51:27 +03:00
arcanis 8c55438140 disable debugpanel for now 2023-05-04 14:30:27 +03:00
arcanis 277d40d231 do not assign path in context manager as it is deprectated 2023-05-04 14:29:39 +03:00
arcanis 2ff56965d9 PEP-673 use Self as return type for classmethods (#94)
* PEP-673 use Self as return type for classmethods

* add dummy test file

* remove python3.10 compat
2023-05-04 03:28:08 +03:00
arcanis 9dc6d56a8d fix flapping tests for oauth
Original implementation sends requests to httpbin which sometimes might
not be available. With proposed changes we are blocking redirects and
just check request itself
2023-05-03 14:53:35 +03:00
arcanis d59a0629c5 PEP-561 complaint: add py.typed marker 2023-04-17 18:56:49 +03:00
arcanis 31f47b8069 fix some pycharm warnings 2023-04-17 02:23:50 +03:00
arcanis e42ca95789 suppress status errors also if option is set 2023-04-15 05:05:55 +03:00
arcanis ff917281a2 add referencne to api docs to docs 2023-04-14 05:06:53 +03:00
arcanis f7e260e1f4 add reference to api documentation from main page 2023-04-12 04:22:49 +03:00
arcanis abf29083c4 PEP-585 complaint: remove type aliases (#93) 2023-04-12 04:09:28 +03:00
arcanis 54d36ccf36 docs update 2023-04-10 21:13:43 +03:00
arcanis 2cd61b0a20 Release 2.8.0 2023-04-09 13:06:54 +03:00
arcanis 9503a9f2ae try to remove unknown packages from api 2023-04-06 18:14:36 +03:00
arcanis 39fde7cd5f hide cookie key and salt from config output 2023-04-06 00:31:50 +03:00
arcanis 58379e7bf3 optimize imports 2023-04-06 00:24:39 +03:00
arcanis 3c068edf4f argument annootation update 2023-04-06 00:24:39 +03:00
arcanis 1106ff6482 fix license url 2023-04-06 00:24:39 +03:00
arcanis e08ab2db10 extract schemas automatically from views 2023-04-06 00:24:39 +03:00
arcanis 8f4a2547e8 use api generated docs instead of comments (#92) 2023-04-06 00:24:39 +03:00
arcanis 7f5e541120 execute request in context methods instead of handling them each time
manually
2023-03-23 12:43:04 +02:00
arcanis ec0550a275 Release 2.7.1 2023-03-06 01:15:47 +02:00
arcanis df23be9269 gracefully terminate web server
In previous revisions server was terminated by itself, thus no lock or
socket was removed. In new version, graceful termination of the queue
has been added as well as server now handles singals
2023-03-06 01:13:41 +02:00
arcanis a8c40a6b87 replace InitializeException with InitializeError in docs 2023-03-02 11:07:59 +02:00
arcanis a274f91677 simplify login ttl processing 2023-02-24 16:52:55 +02:00
arcanis 13faf66bdb add more validation rules 2023-02-23 15:18:56 +02:00
arcanis 4fb9335df9 add ability to read cookie secret from config 2023-02-22 18:47:56 +02:00
arcanis d517d8bfbb Release 2.7.0 2023-02-20 03:05:08 +02:00
arcanis 37e57c13c8 update dependencies before build (#91)
Old implementation has used add step in order to fetch dependencies,
which could lead to build errors in case if dependency list was updated.

New solution uses dependencies which are declared at current version and
fetch them (if required and if enabled) before update process.

Closes #90
2023-02-12 06:02:30 +03:00
arcanis 19bb19e9f5 handle .gitignore file correctly in remote push trigger 2023-02-11 04:41:24 +02:00
arcanis 3a4e8f4d97 mask mypy warning
The newest mypy produces the following warning:

src/ahriman/application/handlers/search.py:43: error: Non-overlapping identity check (left operand type: "Union[_DefaultFactory[Any], Literal[_MISSING_TYPE.MISSING]]", right operand type: "Type[List[Any]]")  [comparison-overlap]

which is more likely caused by updated dataclass models to protoocol (however decorators are still calllable). This commit masks problematic line from checking
2023-02-09 22:46:08 +02:00
arcanis 4db8ad8e8d hide passwords and secrets from repo-config subcommand by default 2023-02-05 16:44:48 +02:00
arcanis 117f096d41 note about local database update (see #85) 2023-01-31 14:47:58 +02:00
arcanis 917ec48be5 handle architecture specific fields for dependencies
This change requires srcinfo at least 0.1.2 version. Unfortunatelly aur
api don't support architecture specific arrays for now, so we just leave
it as is

Closes #82
2023-01-31 14:34:09 +02:00
arcanis 0a2ba4ae07 filter empty packages from database
In some cases (e.g. during addition of the package to build queue) we don't have
full information about package inself; in these cases we produce lines
with empty architecture, which duplicates normal ones.

This commit changes architecture column type to required and also
filters packages which don't have architecture set yet.

Closes #83
2023-01-30 17:57:13 +02:00
arcanis 9d7f63e549 trim version from provides list
Closes #87
2023-01-30 17:27:34 +02:00
arcanis 25eee9ca5e add ability to suppress http logging errors (#86) 2023-01-30 17:19:01 +02:00
arcanis 5af84955ac calculate dependencies based on package information (#89) 2023-01-30 17:28:05 +03:00
arcanis d3ad4c3c08 remove debug line 2023-01-27 16:41:41 +02:00
arcanis 5bbb9d269b Release 2.6.1 2023-01-25 15:28:27 +02:00
arcanis 17466d8d37 make oauth client trully optional (#84)
Same old song, after migraiton to packages some optional modules are
being imported globally which lead to making hard dependency
2023-01-25 15:25:42 +02:00
arcanis 9e4e3b701b enable lock for web service 2023-01-18 01:39:55 +02:00
arcanis c6555cf2c7 Release 2.6.0 2023-01-16 01:35:12 +02:00
arcanis 85baad6a41 add notes about non-x86_64 architecture setup 2023-01-15 18:16:29 +02:00
arcanis 616a1950ae add ability to override pacman mirror in devtools configuration
This commit also extends configuration of the multilib option, adding
the ability to exlcude multilib repository from repositories list

Note, that in order to support repository list and mirror correctly,
alpm configuration section is now architectture specific
2023-01-15 18:05:15 +02:00
arcanis 30b9bcb45a add note about other authroization options to gitremote triggers 2023-01-13 17:27:19 +02:00
arcanis d7356926c4 consider vcs flag and vcs_allowed_age during local packages update 2023-01-12 15:37:05 +02:00
arcanis 43a7d09cab add ability to check for service updates 2023-01-12 15:37:05 +02:00
arcanis 77954b988b subcommand review
Some commands have been moved to another group and thus having another
default name (old subcommands are still available...for now):

* daemon -> repo-daemon
* key-import -> service-key-import
* repo-clean -> service-clean
* repo-config -> service-config
* repo-config-validate -> service-config-validate
* repo-setup -> service-setup
* repo-shell -> service-shell
* version -> help-version
2023-01-12 15:37:05 +02:00
arcanis 0239fb50b6 expose trigger configuration schema
Note that this commit contains the following breaking changes:

* remote pull and remote push triggers are now enabled by default (with
  empty target list)
* remote pull and remote push triggers now require target option to be
  set (old behaviour had fallback on `gitremote`)
* validation is now considered to be stable, so it is enabled by default
  in docker image (can be disabled however)
2023-01-10 03:33:52 +02:00
arcanis d942a70272 add config validator subcommand (#80)
* add config validator subcommand

* add --exit-code flag

* docs & faq update
2023-01-09 18:22:29 +03:00
arcanis 1f07a89316 sort unsafe commands response 2023-01-04 18:00:10 +02:00
arcanis 014007ade3 review unsafe commands access
Some commands were made unsafe in old versions, but nowadays they can be
run without having special privileges.

There was also a bug in which status commands were not available if you
are not ahriman user and unix socket is used. It has been fixed by
switching to manual socket creation (see also
https://github.com/aio-libs/aiohttp/issues/4155)
2023-01-04 17:45:44 +02:00
arcanis 730f3ca0c9 copyright update 2023-01-04 03:43:10 +02:00
arcanis 42c13b5d4b Release 2.5.4 2023-01-03 01:59:25 +02:00
arcanis 04e5a263b7 add notes about documentation and methods inside class
Because I always forget which way I used before
2023-01-03 01:53:10 +02:00
arcanis caca1576c8 Correct way to allow setting context with existing
This reverts commit 5c4d3eeffd.

Original solution has introduced special workaround (strict flag) which
contradicts the concept of immutable context. Moreover, it introduces
possible side-effects, because child process will use the one set by
parent instead of having own one.

The correct solution is to re-create context in process entry point

Sorry, it was Jan 1 and I was drunk :(
2023-01-03 00:48:14 +02:00
arcanis 98f2f19d5b Release 2.5.3 2023-01-02 03:24:11 +02:00
arcanis 5c4d3eeffd allow setting context with existing
In case of running command from web interface, it will raise exception
because context has been copied with subprocesses
2023-01-02 03:21:15 +02:00
arcanis 84d4523e85 Release 2.5.2 2023-01-02 01:57:09 +02:00
arcanis 2c2eae2334 remote all gitfiles in git remote trigger
In case if there is .gitignore file with asterics, the pkgbuild upload
would not appear
2023-01-02 01:45:50 +02:00
arcanis 214d6d7fdd Release 2.5.1 2022-12-31 14:58:37 +02:00
arcanis e9512e9a6a remote log for calculate version as it cleans logs 2022-12-31 14:48:21 +02:00
arcanis f984ea75d0 fully lazy handle load
In case of immediate handle load it would try to sync databases (or at
least to create database files), which is not possible in case if
command is run as non-ahriman user. This commit makes handle load lazy
and allows to run some commands as non-ahriman user
2022-12-31 14:48:21 +02:00
arcanis b44e5c3a3d Release 2.5.0 2022-12-31 02:02:37 +02:00
arcanis 1891d6c944 force upgrade sphinx documentation 2022-12-30 22:58:13 +02:00
arcanis 08d68258ca add completion files generated by shtab 2022-12-30 21:19:23 +02:00
arcanis cd3d12a754 allow 3 symbols for package search 2022-12-30 21:19:23 +02:00
arcanis 81d9526054 improve VCS packages checks
* Unlike older version, currently service will always try to pull AUR
  package to check version. Previously if no-vcs flag is set, it would
  ignore VCS packages completelly
* Introduce build.vcs_allowed_age option. If set, it will skip version
  calculation if package age (now - build_date) is less than this value
2022-12-30 21:19:23 +02:00
arcanis c7447f19f0 change another boolean info option to boolean action 2022-12-29 03:15:48 +02:00
arcanis 7de94a8daa add repo-structure subcommand
This commit also changes Tree class, replacing load method by resolve
2022-12-27 10:35:03 +02:00
arcanis aa1a59d0a3 fix property description in docs 2022-12-26 02:32:14 +02:00
arcanis 8864855c14 write patches via gitremote push trigger (#79)
* write patches via gitremote push trigger

* implement context variables intead of custom database class
2022-12-25 02:10:38 +03:00
arcanis b0b37e8169 Release 2.4.1 2022-12-11 05:17:10 +02:00
arcanis 7df4adfc9d replace alert bodals with toasts 2022-12-09 11:33:19 +02:00
arcanis 2d5b73c6dc remove badges from status page 2022-12-09 00:58:03 +02:00
arcanis 1c8aed4a97 check that package has been created during setup workflow 2022-12-06 11:09:56 +02:00
arcanis 2e059023f3 Release 2.4.0 2022-12-05 00:25:53 +02:00
arcanis da5d97788a do not update database via pacman, use ahriman's own databses 2022-12-04 23:45:10 +02:00
arcanis 0e839fbbf2 Some minor documentation related fixes
* Improve some wording (again)
* Change default type for refresh option to False (does not affect
  behavior)
* Update docstrings to reflect last changes
* Configuration.__convert_path has been replaced by shlex
* aiosecurity functions support kwargs now
2022-12-04 02:10:46 +02:00
arcanis 262462d3c3 improve wording in documentation 2022-12-02 15:45:01 +02:00
arcanis 7aa91f9e2e do not trigger update on sign command 2022-12-02 01:41:23 +02:00
arcanis 01eda513cf improve setup command by --makeflags-jobs argument and fix repository sign on creation 2022-12-02 01:41:23 +02:00
arcanis 0161617e36 implement support of unix socket for server
This feature can be used for unauthorized access to apis - e.g. for
reporting service if it is run on the same machine. Since now it becomes
recommended way for the interprocess communication, thus some options
(e.g. creating user with as-service flag) are no longer available now
2022-12-02 01:41:23 +02:00
arcanis 4811dec759 do not invoke configuration write in case if no salt or user was written 2022-12-02 01:41:23 +02:00
arcanis 09623c20d5 add rebuild implementation to interface 2022-12-02 01:41:23 +02:00
arcanis eba247b759 make package actions as dropdown 2022-12-02 01:41:23 +02:00
arcanis 5073c80af1 add key-import button to interface 2022-12-02 01:41:23 +02:00
arcanis 766081d212 add demos links 2022-11-24 02:38:33 +02:00
arcanis 896cd0bd71 add security notes 2022-11-24 02:38:33 +02:00
arcanis ce1bd2f2db add curl examples to web views 2022-11-24 02:38:33 +02:00
arcanis 6ba96d838d build docs together with web views 2022-11-24 02:38:33 +02:00
arcanis df9e03f495 change respone for service requests 2022-11-24 02:38:33 +02:00
arcanis 89944eb2b6 add fallback for copying to clipboard 2022-11-24 02:38:33 +02:00
arcanis 336784519b add show/hide password button 2022-11-24 02:38:33 +02:00
arcanis f3341ec7cd update web preview picture 2022-11-24 02:38:33 +02:00
arcanis d36e851a29 render httpunauthorized as html in plain http requests 2022-11-22 23:40:27 +02:00
arcanis 28bd5f2095 allow scrolling with fixed copy button position 2022-11-22 22:20:21 +02:00
arcanis 011b4e2e31 change logging module imports 2022-11-22 22:20:21 +02:00
arcanis 15609ba044 change wording for package actions 2022-11-22 15:46:49 +02:00
arcanis 6d4f9981f7 fix login and logout buttons decorations 2022-11-22 11:17:59 +02:00
arcanis bbb97d1cdd add more notes about docker 2022-11-22 10:49:46 +02:00
arcanis 3bca780bdd check log record in handler instead of client 2022-11-22 02:30:37 +02:00
arcanis 78e6b48c24 ask users to repeat password
In case if password is asked via getpass, it is possible to make typo
and user will not see the mistake. In order to avoid it, additional
confirmation has been added
2022-11-22 02:19:37 +02:00
arcanis 137d62e2f8 Extended package status page (#76)
* implement log storage at backend
* handle process id during removal. During one process we can write logs from different packages in different times (e.g. check and update later) and we would like to store all logs belong to the same process
* set package context in main functions
* implement logs support in interface
* filter out logs posting http logs
* add timestamp to log records
* hide getting logs under reporter permission

List of breaking changes:

* `ahriman.core.lazy_logging.LazyLogging` has been renamed to `ahriman.core.log.LazyLogging`
* `ahriman.core.configuration.Configuration.from_path` does not have `quiet` attribute now
* `ahriman.core.configuration.Configuration` class does not have `load_logging` method now
* `ahriman.core.status.client.Client.load` requires `report` argument now
2022-11-22 02:58:22 +03:00
arcanis 8a6854c867 Release 2.3.0 2022-11-16 00:53:48 +02:00
arcanis 299732181c remote threadname from logging
Since application is mostly singlethreaded it makes no sense to log it
2022-11-14 23:21:06 +02:00
arcanis 84c1b4d82d Release 2.3.0rc4 2022-11-14 01:02:27 +02:00
arcanis cdd66ee780 fix case when no files were commited in remote push trigger
The issue appears together with --intent-to-add flag for adding new
files. Original testing has been performed by having already added new
files, thus it passed all checks.

This commit also adds `commit_author` option which will allow to
overwrite the author.
2022-11-14 00:59:43 +02:00
arcanis b2ed383de0 Release 2.3.0rc3 2022-11-11 21:20:55 +02:00
arcanis 551ee670bf rollback cwd parameter for clone 2022-11-11 21:19:27 +02:00
arcanis 5d4bd9e459 Release 2.3.0rc2 2022-11-11 17:25:11 +02:00
arcanis 4f21eb6fe6 Fix issue when there is no cached source directory yet (closes #75) 2022-11-11 17:23:15 +02:00
arcanis 9a008ddafa Release 2.3.0rc1 2022-11-11 16:36:13 +02:00
arcanis 0cd07afa0f use intersection of from_database and depends_on filters for the rebuild subcommand
Old logic used OR condition, i.e. if set from-database, it would ignore
the --depends-on flag. In new logic it calculates dependencies based on
the package list, which can be retrieved from database
2022-11-11 16:02:19 +02:00
arcanis f590136197 limit max module size and improve some help messages 2022-11-11 16:01:54 +02:00
arcanis 1855e513a8 speedup docker build command by adding dockerignore listing 2022-11-11 12:47:31 +02:00
arcanis 233dda8420 man update 2022-11-10 20:10:07 +02:00
arcanis 9556779ee2 add exec to shell subcommand 2022-11-10 18:52:55 +02:00
arcanis 8b32bd43db make exception optional argument for check_output method 2022-11-10 18:05:32 +02:00
arcanis 0eadef597a replace several store_true keys to booleanoptionalaction alternative (#74) 2022-11-10 18:34:01 +03:00
arcanis e58ccdc8ad write .makepkg.conf to home dir instead of repository root (#72) 2022-11-08 17:02:13 +02:00
arcanis 2a07356d24 add ability to use ahriman pacman database instead of system one (#71)
By default this feature is enabled. On the first run it will copy (if
exists) databases from filesystem to local cache (one per each
architecture). Later it will use this cache for all alpm operations. In
order to update this cache, some commands (mainly package building)
provide `-y`/`--refresh` option which has same semantics as pacman -Sy
does.

Note however that due to extending `Pacman` class some methods were
renamed in order to be more descriptive:
* `Pacman.all_packages` -> `Pacman.packages`
* `Pacman.get` -> `Pacman.package_get`

This commit also adds multilib repository to the default docker image
which was missed.
2022-11-08 17:26:51 +03:00
arcanis 43c553a3db improve repo-setup command
* Move devtools executable to ahriman home, because we don't really
  need to use executable inside root
* Use named sudoers file instead of single file. It will allow easily to
  remove file as well as use setup command for several
  repositories/architectures
2022-11-06 14:25:44 +02:00
arcanis 0d1a890bd0 fix case in which if no source PKGBUILDs were updated, an exception will occur 2022-11-03 03:39:54 +02:00
arcanis 3aeaa8ce2c faq update 2022-11-02 17:15:17 +02:00
arcanis 7cee859f19 docs update 2022-11-02 04:17:26 +02:00
arcanis fb6b22cdd7 change gitremote api to be same as report and upload
These changes are keeping fallback to old settings, but will allow to
run multiple git targets with different settings
2022-11-02 04:09:42 +02:00
arcanis 4f35ec6542 fix update handler tests
If daemon tests are run before the update handler it causes test
failure, because there are other calls
2022-10-31 11:51:12 +02:00
arcanis d98211e5e5 add possibility to run full update
In case if packages are not set from web, the spawner will run full
repository update
2022-10-31 02:41:24 +02:00
arcanis b97c8928e1 add daemon subcommand
This command emulates default systemd timer and can be useful in docker
container in order to run 24/7
2022-10-31 01:38:01 +02:00
arcanis 649df81aa5 implement single-function patches (#69) 2022-10-30 03:11:03 +03:00
arcanis ad7cdb7d95 drop ahriman.core.triggers.Trigger.run method
In order to force new triggers to use on_result method, the old method
has been removed. However, default on_result method still checks if the
old method exists and tries to run it
2022-10-19 20:07:31 +03:00
arcanis 4bb598d2eb fix rtd docs building
Commit 6633766cc3 introduced kw_only
dataclasess which require python 3.10+
2022-10-18 02:24:18 +03:00
arcanis f47be6cab0 disallow no values in configuration
This option could lead to missing warnings about missing or invalid
configuration values because code usually expects that values are exists
and not empty unless it is explicitly specified.

However, pacman configuration still requires this option in order to be
able to deal with boolean values
2022-10-18 02:13:01 +03:00
arcanis 342b3cb652 Add gitremote triggers (#68)
* add gitremote pull trigger

* add push gitremote trigger

* docs update
2022-10-18 01:46:27 +03:00
arcanis fc0d8387df extend triggers to on_start and on_stop methods
This commit also replaces old run method to new on_result
2022-09-26 01:22:54 +03:00
arcanis e0b0c3caeb add minimal install step to workflow 2022-09-17 14:52:08 +03:00
arcanis 61969dd682 make sqlite import consistent 2022-09-17 14:32:21 +03:00
arcanis e441d93a56 Release 2.2.2 2022-09-17 04:05:06 +03:00
arcanis 664b6369bb skip architecture list patching in case if any architecture is set 2022-09-17 04:04:28 +03:00
arcanis 4f6bd29ff4 Release 2.2.1 2022-09-14 04:49:08 +03:00
arcanis f6d9ea480a docs update 2022-09-14 04:48:11 +03:00
arcanis 08f62842ba Release 2.2.0 2022-09-14 03:28:28 +03:00
arcanis 1912790ccc Make optional dependencies trully optional (#67)
The issue appears when there is no boto, jinja and some other libraries
are not installed because the classes which use these libraries are
still being imported inside the package file. The fix removes those
imports from package root, because they should not be here, in fact,
content of report and upload packages must be imported only inside the
trigger class and only if they are actually required

This commit also adds setuptools as required dependency since it is used
for some parsers (previously it was provided dependency)
2022-09-11 01:44:06 +03:00
arcanis cf3c48ffeb patch architecture list in runtime (#66) 2022-08-09 15:18:20 +03:00
arcanis 6633766cc3 frozen dataclasses 2022-07-26 14:40:28 +03:00
arcanis f73d1eb424 reduce docker image size a bit 2022-07-18 11:42:26 +03:00
arcanis 87ab3683ad Release 2.1.0 2022-07-02 15:42:39 +03:00
arcanis 9c2f73af8c simplify tmpdir method 2022-06-28 19:11:38 +03:00
arcanis 31f551bdf2 review loggers once more
This commit makes loggers like java.util.logging with fully qualified
logger name which is created by LazyLogging trait
2022-06-28 11:00:45 +03:00
arcanis cbbe2b01e4 fix case with package name which cannot be downloaded
(without special settings)

The issue appears if file or its version contains one of special URI
characters, e.g. +. Theu will be interpreted as query parameters by
(some) servers (e.g. S3 works in this way). In this commit we rename
archive to the one with safe name.
2022-06-27 18:53:48 +03:00
arcanis cd361a483d review loggers once more
This commit makes loggers like java.util.logging with fully qualified
logger name which is created by LazyLogging trait
2022-06-27 01:41:49 +03:00
arcanis cee4fd4cce bump web libraries
also encode strings for the views
2022-06-24 12:03:38 +03:00
arcanis 18daecaac7 review loggers 2022-06-21 11:48:42 +03:00
arcanis 03c298c762 replace if with while for telelgram reporting 2022-06-10 12:02:00 +03:00
arcanis 79d4a488a0 Release 2.0.0 2022-06-05 20:46:32 +03:00
arcanis 22847aac68 small docs update 2022-06-04 15:24:14 +03:00
arcanis 75b9a96798 do not pass timestamp to pretty_datetime functioon 2022-05-31 21:26:39 +03:00
arcanis a0cfb7561d rewrite api under single /api endpoint 2022-05-29 17:15:49 +03:00
arcanis f4131b8cd7 add shell and version parser 2022-05-27 18:34:07 +03:00
arcanis 9f134e37b6 update readme 2022-05-22 16:00:24 +03:00
arcanis 07b7689701 do not bump timestamp on empty updates 2022-05-21 04:19:04 +03:00
arcanis 47de715d7d dynamic html load (#63)
* dynamic html load
* split by classes
2022-05-20 22:29:36 +03:00
arcanis 5674b7b388 add ability to run only speicifed triggers from command line
This commit also restores repo-report and repo-sync subcommands
2022-05-12 18:43:36 +03:00
arcanis f6e57a411c fix case when provided trigger path is directory
If trigger is directory and we have permissions to read it, the loaded
will try to load this from it and will fail with IsADirectoryError
2022-05-12 07:22:50 +03:00
arcanis 38e48b1571 add not about pacman database update 2022-05-11 04:30:24 +03:00
arcanis d492c968ea add contribution guide 2022-05-11 03:17:37 +03:00
arcanis a11fd188a2 Release 2.0.0rc11 2022-05-10 06:03:33 +03:00
arcanis 2431d5de0e fix bug with checking file
The bug appear when the file exists or doesn't, but we don't have
permissions to read it. This one must be treated as missed permission
2022-05-10 06:01:41 +03:00
arcanis 88f71b240d Release 2.0.0rc10 2022-05-09 21:51:35 +03:00
arcanis 99874845b5 triggers implementation (#62) 2022-05-09 20:00:20 +03:00
arcanis d98cfa3732 Release 2.0.0rc9 2022-05-08 03:58:53 +03:00
arcanis b6db2a8035 fix error with missing sources
In case if package has local cache it will fail to load because no
remote source set. Particially this case can be observed during tree
load
2022-05-08 03:56:54 +03:00
arcanis 47c578ea08 Release 2.0.0rc8 2022-05-06 20:55:54 +03:00
arcanis 98910240dd shorten public imports 2022-05-06 04:08:05 +03:00
arcanis 33e9fea47c Docs update (#61)
* Improve sphinx documentation

* update faq formatting

* fix setup doc

* fix docs according to the generated htmls
2022-05-06 02:54:37 +03:00
arcanis 304690e0d0 correct archllinux wording in readme 2022-05-06 02:54:37 +03:00
arcanis 3d9fae5415 add __all__ attribute 2022-05-06 02:54:37 +03:00
arcanis b7dca2d797 add note about ahriman to index page 2022-05-06 02:54:37 +03:00
arcanis b7debddaea Complete official repository support (#59) 2022-05-03 00:49:32 +03:00
arcanis 1cfc751d21 rename classes in order to make documentation reference work 2022-04-18 05:30:02 +03:00
arcanis 6ebbb04504 readthedoc integration 2022-04-18 04:39:40 +03:00
arcanis c9ee470ee2 move documentation to tox 2022-04-18 01:48:36 +03:00
arcanis a2610504e5 apply data migration in the same transaction block with schema migration 2022-04-18 01:19:38 +03:00
arcanis 36b8b0f46a fix target naming 2022-04-18 01:19:20 +03:00
arcanis d90f417cae Docstring update (#58)
* migrate docstrings from reST to google format

* add raises note

Also change behaviour of the `from_option` method to fallback to
disabled instead of raising exception on unknown option

* fix part of warnings for sphinx

* make identation a bit more readable

* review fixes

* add verbose description for properties to make them parsed by sphinx extenstion

* add demo sphinx generator
2022-04-17 20:25:28 +03:00
arcanis 0db619136d Release 2.0.0rc7 2022-04-11 00:48:08 +03:00
arcanis 208a9b920d docs update 2022-04-11 00:46:46 +03:00
arcanis cb63bc08ff add backup and restore subcommands 2022-04-10 21:34:34 +03:00
arcanis 6551c8d983 merge restore to rebuild commannd 2022-04-10 01:51:12 +03:00
arcanis a6c8d64053 Release 2.0.0rc6 2022-04-09 17:34:23 +03:00
arcanis fd78f2b5e2 do not render failed packages in jinja (#57)
basic templates require package info which is unavailable if package
wasn't built
2022-04-09 17:31:13 +03:00
arcanis 900907cdaa Release 2.0.0rc5 2022-04-08 04:42:05 +03:00
arcanis 5ff2f43506 change telegram default index to telegram-index 2022-04-08 04:32:34 +03:00
arcanis dd521b49b5 force git run from the same dir to clone 2022-04-08 04:04:06 +03:00
arcanis 5b1f5a8473 fix users migration 2022-04-08 03:45:17 +03:00
arcanis 86af13f09e add telegram integraion 2022-04-08 03:41:07 +03:00
arcanis 733c014229 Release 2.0.0rc4 2022-04-08 01:14:35 +03:00
arcanis 783c16b2ed trim versions before dependency list calculation
When dependencies list contains same package with version it tries to
find packages which don't exists
2022-04-07 20:32:55 +03:00
arcanis 2536b8dc1f add support of repository restoration 2022-04-07 04:49:07 +03:00
arcanis e200ac9776 add support of officiall repositories api 2022-04-07 04:19:37 +03:00
arcanis 6946745153 fix descriptions 2022-04-06 01:48:03 +03:00
arcanis 6de75377c3 Release 2.0.0rc3 2022-04-04 02:40:17 +03:00
arcanis a734b86e66 allow any tag for push 2022-04-04 02:39:56 +03:00
arcanis 74906d084a Release 2.0.0rc2 2022-04-04 02:27:19 +03:00
arcanis 22d1d835af change regex for sed 2022-04-04 02:26:42 +03:00
arcanis b7cd83502a Release 2.0.0-rc1 2022-04-04 02:18:31 +03:00
arcanis 2d2cd73010 better processing for subcommands
Old versions cached full output to memory and only after that printed it
into log. This behaviour causes issues in case if operation stucks and
you would need to find the step at which it does. New check_output
method uses Popen directly and iterates over stdout lines

Also changed behaviour from merging stderr into stdout to separate
stderr logging.

Any other behaviour of the function must be the same.

Also changed GPG.key_import method to disable local signing since it
seems it is useless (and may break process in case if there is no
private key)
2022-04-04 00:03:22 +03:00
arcanis 40b84b810d docs update 2022-04-03 01:44:11 +03:00
arcanis 7188cfaf78 remove implicit user from docker 2022-04-01 18:39:24 +03:00
arcanis e909e4f570 add configurable exit codes to some commands (#55) 2022-04-01 18:30:11 +03:00
arcanis 06ec16ac77 create user in docker container
also fix user reading and add nocolor flag by default
2022-03-31 03:25:53 +03:00
arcanis fed2f3aab8 configure docker to work with tags 2022-03-31 02:19:48 +03:00
arcanis 2bc2b63a8a add help command 2022-03-31 02:18:39 +03:00
arcanis 0c5a90cd66 update license years 2022-03-31 01:49:31 +03:00
arcanis 28cc38aaa5 port part of settings to database (#54) 2022-03-31 01:48:06 +03:00
arcanis fb02e676af migrate to tox
this also requires to move default configuration files to share/ahriman.
Thus the following features have been added
* default configuration is not stored in /usr/share/ahriman/settings
* package installed via PKGBUILD now copies files from /usr
* configuration class now fallbacks to default in /usr
2022-03-21 04:38:42 +03:00
arcanis 13121298f5 deprecate init/repo-init command
In current workflow you need to run setup to run init (because of
repository name), but you need to run init before setup (because of
repository tree rights).

New solution just add `Repo.init()` method call to setup subcommand
after config reload to make sure that repository name has been applied.
In addition chown method as well as setuid method for check_output have
been added.
2022-03-21 01:32:11 +03:00
arcanis 63e79ec57a enable privileged mode for the docker 2022-03-20 04:05:43 +03:00
arcanis b83bc5a732 fallback to AUR source in case of PermissionError exception 2022-03-20 03:46:56 +03:00
arcanis 8374590b62 add fallback for utf option 2022-03-20 03:16:33 +03:00
arcanis 4cc665f6ee replace pspec with spec 2022-03-20 03:08:51 +03:00
arcanis 57f25c309a add console printer
also add python-requests as explicit dependency and escape symbols in
repository name for badges in default tempate
2022-03-20 02:31:50 +03:00
arcanis d33844e7b2 ALLOW_AUTO_ARCHITECTURE_RUN flag for unsafe commands 2022-03-19 03:08:45 +03:00
arcanis e9886efaa2 docker hub integration 2022-03-14 00:08:21 +03:00
arcanis 61efbb71a2 add docker support (#52)
* add docker support

* make shellcheck happy
2022-03-13 23:43:25 +03:00
arcanis 9f4acacada strict test checks 2022-03-04 02:04:52 +03:00
arcanis 026f74121a set success status on up to date packages 2022-02-06 06:22:58 +03:00
arcanis e414616bbd Release 1.8.0 2022-02-06 05:45:50 +03:00
arcanis 60a2e25b9a update for new aiohttp api 2022-02-06 04:05:33 +03:00
arcanis 683abca9e5 use own aur wrapper (#49) 2022-02-06 03:44:57 +03:00
arcanis 5a3770b739 Release 1.7.0 2021-12-26 02:01:09 +03:00
arcanis 52cd9a0ea9 make mypy happy 2021-12-26 01:58:55 +03:00
arcanis bfca7e41ab handle dependencies recursively 2021-12-22 19:35:09 +03:00
arcanis 603c5449a8 initial implementation of the local git clones (#48) 2021-12-22 15:56:24 +03:00
arcanis 5aac3db2d5 do not read aur_url from settings, use repository property instead 2021-11-15 11:27:41 +03:00
arcanis 3c5bcbd172 Release 1.6.4 2021-11-10 21:29:45 +03:00
arcanis 042638d40e handle packages which have been removed from the repository (#45)
* handle packages which have been removed from the repository

* manually remove packages which have been removed from the base
2021-11-10 01:37:25 +03:00
arcanis e6adb333b2 Release 1.6.3 2021-11-04 21:32:27 +03:00
arcanis fa4244d21e take python laziness into account 2021-11-04 21:30:34 +03:00
arcanis 91de1c2b8a Release 1.6.2 2021-10-28 03:20:52 +03:00
arcanis 32a4a82603 improve configuration extension
* Allow spaces in lists. This feature has been done in the way as shell
  interprets arguments by using quotation marks
* Clear current content on reload
2021-10-28 03:19:50 +03:00
arcanis e8a10c1bb5 add nginx configuration to the faq 2021-10-27 03:35:33 +03:00
arcanis d480eb7bc3 Release 1.6.1 2021-10-27 03:16:53 +03:00
arcanis 8b0f9bfd78 update license headers 2021-10-27 03:14:39 +03:00
arcanis a2639f8dbb add update printer which will print current version if any 2021-10-27 03:11:43 +03:00
arcanis 65ba590ace use PackageSource enum for Package.load method
When using add function it sill tries to load data with invalid source
2021-10-27 02:49:23 +03:00
arcanis fcb130e226 Release 1.6.0 2021-10-27 01:59:36 +03:00
arcanis ae99fe4535 drop no-quiet option and change tree_create message error to warn 2021-10-27 01:57:54 +03:00
arcanis ec23e3f912 remove help sample from readme because it changes faster than om able to maintain it 2021-10-26 04:53:45 +03:00
arcanis d3ea81d234 unify aur.search method
due to specific of the AUR API in order to reduce the code we are using
own wrapper and work with it instead of direct library calls
2021-10-26 04:49:55 +03:00
arcanis 09b0f2914d Add ability to show more info in search and status subcommands
This feature also introduces the followiing changes
* aur-search command now works as expected with multiterms
* printer classes for managing of data print
* --sort-by argument for aur-search subcommand instead of using package
  name
* --quiet argument now has also --no-quite option
* if --quite is supplied, the log level will be set to warn instead of
  critical to be able to see error messages
* pretty_datetime function now also supports datetime objects
* BuildStatus is now pure dataclass
2021-10-26 04:27:36 +03:00
arcanis 7351e20104 always update environnment before any action 2021-10-24 04:14:57 +03:00
arcanis dfd87c502f split application class into traits 2021-10-23 13:44:57 +03:00
arcanis 0b9ab09879 add patches to clean command 2021-10-20 03:22:16 +03:00
arcanis 47c54f0b40 add ability to download package from external links (e.g. HTTP) 2021-10-20 03:09:58 +03:00
arcanis a2f2fa0354 add ability to read argument list from file 2021-10-20 02:15:59 +03:00
arcanis 4d68080c05 logger improvements
* remove build log since it has no usages actually (replaced by root
  logger)
* decrease boto3 log levels to INFO by default to reduce noice
2021-10-20 02:12:49 +03:00
arcanis eb16ef12f3 always return json in responses 2021-10-20 02:12:39 +03:00
arcanis e10e362dae Release 1.5.0 2021-10-18 03:48:24 +03:00
arcanis e59fdd1ccc minor architecture description update 2021-10-18 03:35:39 +03:00
arcanis 22d92e3b4e add repo-status-update subcommand 2021-10-17 06:38:49 +03:00
arcanis 56b77a84a6 allow to use multiple upload and report targets with the same name
In this feature target option must allways point to section name instead
of type. Type will be read from type option. In case if type option is
not presented it will try to check if section with architecture exists
(e.g. target = email, section = email:x86_64); if it does, the correct
section name and type will be used. Otherwise it will check if the
specified section exists; if it does, seection name and type will be
returned.
2021-10-17 06:06:08 +03:00
arcanis a5a99ec0b8 split github upload into generic http method and github specific
We might use some features from the http upload for another parser
2021-10-15 23:36:26 +03:00
arcanis 04bbabe898 docs update 2021-10-15 04:55:46 +03:00
arcanis 4521c2adde disallow to create tree in case of unsafe run 2021-10-14 04:53:09 +03:00
arcanis 5c5e54228f use generic removal method 2021-10-14 04:08:21 +03:00
arcanis 6514924b2d change method spelling
in order to sort method correctly we are going to use the following
namiing schema:

{subject}_{action}_{details}

This schema still have some exceptions, e.g. single word methods, bool
methods (is_) and getters in case if they are singular (i.e. there is
no any other method with this subject)
2021-10-14 04:01:54 +03:00
arcanis 16aa977fa8 add test for every file 2021-10-14 03:34:12 +03:00
arcanis 6e377e7261 aggressive small case 2021-10-14 03:13:15 +03:00
arcanis 4502931c39 exactly one called with instead of last call check 2021-10-14 03:12:45 +03:00
arcanis fcb167b1a3 github upload support (#41) 2021-10-14 02:30:13 +03:00
arcanis 72b26603bf add ability to add manually stored packages (#40)
* add ability to add manually stored packages

* update tests

* handle manual packages in remove-unknown method

* live fixes

also rename branches to has_remotes method and change return type
2021-10-12 21:15:35 +03:00
arcanis ab8ca16981 improve scripts
move logic to separated shell scripts and also create shell script for
repository setup

Also force create directory according to systemd recommendations
2021-10-11 02:20:16 +03:00
arcanis 7c4f84fbc7 more verbose help messages 2021-10-07 01:56:22 +03:00
arcanis 3b6b2efcb1 patch control subcommands 2021-10-05 08:57:42 +03:00
arcanis 9f99dd3ff2 patch support (#35) 2021-10-03 15:20:36 +03:00
arcanis bee97df87f do not write anything on httpexceptions in log 2021-10-03 02:05:28 +03:00
arcanis 6becd01803 replace no-log with quiet
Also behavior of the flag has been changed: now it disables logs at all
2021-10-03 01:59:33 +03:00
arcanis db195391e4 Release 1.4.1 2021-10-03 01:02:41 +03:00
arcanis 59f2992559 do not use set_defaults for architecture arguments
according to the source code defaults always updates the values
dictionary. This in this specific case it is impossible to override the
value it will be always empty list.

In order to handle it we are adding another property to the Handler
class which allows to run with None architecture list.

This particular set_defaults behaviour is still useful for other cases
when we have to run command without any specific architecture
2021-10-03 00:59:24 +03:00
arcanis 4f06647193 Release 1.4.0 2021-10-01 09:25:35 +03:00
arcanis 73a4cee257 add package request endpoint 2021-10-01 08:58:50 +03:00
arcanis 13d00c6f66 docs update 2021-09-26 14:39:35 +03:00
arcanis 3e032c3515 add index url 2021-09-26 14:29:42 +03:00
arcanis d73d5daad3 add debugtoolbar support 2021-09-26 12:31:12 +03:00
arcanis f55b44b391 set both value and innerText during search
current implementation just adds package name with the description which
is incorrect
2021-09-26 10:34:21 +03:00
arcanis 51b28baf40 add ability to specify package source explicitly during the addition 2021-09-26 09:55:14 +03:00
arcanis 24326f9753 define permissions in views directly 2021-09-25 17:03:46 +03:00
arcanis 36c763069d only set file rights if requested 2021-09-23 20:36:22 +03:00
arcanis c9a155bbc4 raise httpexception instead of returning it from a function 2021-09-19 14:28:24 +03:00
arcanis 182bde5e09 add manpage 2021-09-19 13:56:11 +03:00
arcanis 799572fccf Release 1.3.0 2021-09-18 06:28:52 +03:00
arcanis a7a32f0080 better reload 2021-09-17 19:15:53 +03:00
arcanis af3afecce8 update aioauth-client to newest version 2021-09-17 18:45:43 +03:00
arcanis 16bb1403a1 add ability to reload authentication module 2021-09-17 16:05:38 +03:00
arcanis 41731ca359 add ability to remove an user
also replace old user by new one before creation
2021-09-16 02:41:56 +03:00
arcanis e99c2b0c83 remove own implementations of getlist and getpath method in order to use
converters feature
2021-09-14 03:57:20 +03:00
arcanis 6294c0ba14 add ability to filter status response by package status 2021-09-13 23:27:36 +03:00
arcanis 2c74be31bd raise InvalidCommand exception in case if remove option supplied without
package
2021-09-13 22:38:38 +03:00
arcanis 0744ee53dc change spelling for distro name 2021-09-13 02:49:22 +03:00
arcanis 284fd759bf add target for architecture and also update it 2021-09-13 02:01:48 +03:00
arcanis 6f5b28c4f8 expiration on server side support (#33) 2021-09-13 01:18:04 +03:00
arcanis d211cc17c6 send data in json_responses in case of error instead of text 2021-09-12 22:41:02 +03:00
arcanis 117e69c906 return description from the search 2021-09-12 22:31:34 +03:00
arcanis d19deb57e7 OAuth2 (#32)
* make auth method asyncs

* oauth2 demo support

* full coverage

* update docs
2021-09-12 21:41:38 +03:00
arcanis 1b29b5773d remove deprecated access status 2021-09-11 23:06:30 +03:00
arcanis 8e14e8d2cb add error description to modals 2021-09-11 23:05:51 +03:00
arcanis 875bfc0823 add static files support and cookie expiration settings 2021-09-11 16:34:43 +03:00
arcanis 7abdb48ac0 documentation update 2021-09-10 03:32:45 +03:00
arcanis 98eb93c27a Add ability to trigger updates from the web (#31)
* add external process spawner and update test cases

* pass no_report to handlers

* provide service api endpoints

* do not spawn process for single architecture run

* pass no report to handlers

* make _call method of handlers public and also simplify process spawn

* move update under add

* implement actions from web page

* clear logging & improve l&f
2021-09-10 00:33:35 +03:00
arcanis 18de70154e add option to set user-password for service when updating its hash 2021-09-05 15:40:03 +03:00
arcanis 08e0237639 move api endpoints to status-api 2021-09-05 06:46:14 +03:00
arcanis 891c97b036 allow head for every get request 2021-09-05 06:41:50 +03:00
arcanis 55c3386812 add repository name to base tempalte 2021-09-05 06:15:24 +03:00
arcanis b0575ee4ba allow read only pages to be requested without authorization 2021-09-05 06:09:43 +03:00
arcanis e0607ba609 update docs 2021-09-05 05:38:45 +03:00
arcanis 9b8c9b2b2d migration of jinja tempaltes to bootstrap (#30) 2021-09-05 05:27:58 +03:00
arcanis ecf45bc3bb add remove uknown method (#29) 2021-09-03 02:28:27 +03:00
arcanis aecd679d01 add license header to __init__.py 2021-09-02 23:43:05 +03:00
arcanis e63cb509f2 Auth support (#25)
* initial auth implementation

* add create user parser

* add tests

* update dependencies list

* add login annd logout to index also improve auth

* realworld fixes

* add method set_option to Configuration and also use it everywhere
* split CreateUser handler to additional read method
* check user duplicate on auth mapping read
* generate salt by using passlib instead of random.choice
* case-insensetive usernames
* update dependencies
* update configuration reference
* improve tests

* fix codefactor errors

* hide fields if authorization is enabled, but no auth supplied

* add settings object for auth provider

* readme update
2021-09-02 23:36:00 +03:00
arcanis 3922c55464 use {} instead of dict literal according to pylint recommendation 2021-08-22 13:24:42 +03:00
arcanis 9d2a3bcbc1 Release 1.2.6 2021-08-21 16:19:21 +03:00
arcanis a5455b697d Release 1.2.5 2021-08-19 02:36:05 +03:00
arcanis 0bfb763b2a disable manpages building for now since it requires installed distribution 2021-08-19 02:35:48 +03:00
arcanis 9f3566a150 Release 1.2.4 2021-08-19 00:45:58 +03:00
arcanis 16a6c4fdd7 include setup.cfg to tarball 2021-08-19 00:45:26 +03:00
arcanis 91f66fdcee Release 1.2.3 2021-08-19 00:18:12 +03:00
arcanis bb45b1d868 split S3.sync to different methods 2021-08-18 23:59:18 +03:00
arcanis 3d10fa472b guess mime type for local files 2021-08-18 05:04:26 +03:00
arcanis a90c93bbc4 add manpage generator 2021-08-17 04:05:18 +03:00
arcanis 41a3c08d9f Release 1.2.2 2021-08-17 01:03:04 +03:00
arcanis cb328ad797 fix typo in log naming 2021-08-17 01:02:42 +03:00
arcanis 810091cde9 Release 1.2.1 2021-08-17 00:52:09 +03:00
arcanis fc0474fa8f logging rethink
* well lets replace f-strings by %s as it is originally recommended
* use syslog handler by default
2021-08-17 00:23:34 +03:00
arcanis b94179e071 use asyncmock from unittest library 2021-08-11 21:09:10 +03:00
arcanis 9c5a9f5837 Release 1.2.0 2021-08-11 05:02:23 +03:00
arcanis 83047d8270 cleanup and speedup runs 2021-08-11 04:59:45 +03:00
arcanis 990d5dda81 use nosec instead of disabling mktemp rule 2021-08-11 02:51:29 +03:00
arcanis 48e79ce39c add bandit integration and fix its warnings 2021-08-11 02:45:13 +03:00
arcanis 375d7c55e5 web templates improvements
* enable jinja autoescape by default for jinja raw generator
* allow to search by multiple strings (OR)
* replace test templates by symlink
2021-08-11 02:12:14 +03:00
arcanis db52b8e844 move web server to loopback by default 2021-08-11 02:02:18 +03:00
arcanis 50af309c80 add docstrings for every fixture and test methods
also add tests for missing components
2021-08-11 01:55:27 +03:00
arcanis 581401d60f skip update process if no update supplied 2021-08-10 23:25:12 +03:00
arcanis c2685f4746 Native s3 sync (#23)
* Native S3 sync implementation

* fix imports

* fix paths reading

* install s3 components duriing test stage
2021-08-10 23:18:56 +03:00
arcanis 952b55f707 Release 1.1.0 2021-07-05 22:11:14 +03:00
arcanis b9b012be53 handle provides list 2021-07-05 22:08:04 +03:00
arcanis b8036649ab install types for mypy 2021-06-28 02:54:20 +03:00
arcanis c90e20587e remove type: ignore for newest python 2021-06-28 02:32:54 +03:00
arcanis 3e020ec141 Feature/all archs (#21)
* add init subcommand

* add also init command to repository object

* add ability to generate list of architectures

* check if architecture list is not empty
2021-05-23 16:40:40 +03:00
arcanis 783b7d043d imply no-log for every unsafe parser (#20) 2021-05-19 23:30:59 +03:00
arcanis 5c297d1c67 allow to specify list of package dependencies in rebuild target
also replace nargs= by action=append in non-positional args. It is
required to make arguments parsing result more predictable and
consistent
2021-04-18 13:34:27 +03:00
arcanis b0d1f3c091 Release 1.0.0 2021-04-10 01:38:55 +03:00
arcanis 50e219fda5 import pgp key implementation (#17)
* import pgp key implementation

* do not ask confirmation for local sign. Also add argparser test

* superseed requests by python-aur package

* ...and drop --skippgpcheck makgepkg flag by default
2021-04-10 01:37:45 +03:00
arcanis 75298d1b8a better naming for actions 2021-04-09 20:02:17 +03:00
arcanis 8196dcc8a0 add search subparser (#15) 2021-04-09 11:57:06 +03:00
arcanis f634f1df58 Add web status route (#13)
* add status route

* typed status and get status at the start of application
2021-04-08 01:48:53 +03:00
arcanis 32df4fc54f Move search line inside extended report option 2021-04-06 17:03:34 +03:00
arcanis 11ae930c59 Release 0.22.1 2021-04-06 05:54:04 +03:00
arcanis 9c332c23d2 format long line 2021-04-06 05:53:38 +03:00
arcanis 4ed0a49a44 add ability to skip email report generation for empty update list 2021-04-06 05:51:50 +03:00
arcanis 50f532a48a Release 0.22.0 2021-04-06 05:46:12 +03:00
arcanis c6ccf53768 Email report (#11)
* Demo email report implementation

* improved ssl mode

* correct default option spelling and more fields to be hidden for not
extended reports
2021-04-06 05:45:17 +03:00
arcanis ce0c07cbd9 Release 0.21.4 2021-04-05 02:28:38 +03:00
arcanis 912a76d5cb drop changelog
the main reason is that it uses github to generate changelog. Thus it
will be updated AFTER release is created
2021-04-05 02:27:12 +03:00
arcanis 76d0b0bc6d Release 0.21.3 2021-04-05 02:22:44 +03:00
arcanis 27d018e721 update changelog at correct step
also fix commit filter and do not update sha anymore
2021-04-05 02:22:11 +03:00
arcanis a0e20ffb77 Release 0.21.2 2021-04-05 02:01:28 +03:00
arcanis 96e4abc3c0 add changelog generator to both gh-actions and repository 2021-04-05 02:00:05 +03:00
arcanis 6df60498aa Release 0.21.1 2021-04-05 00:45:12 +03:00
arcanis eb0a4b6b4a use globing instead 2021-04-05 00:44:39 +03:00
arcanis 8f469e7eac Release 0.21.0 2021-04-05 00:38:23 +03:00
arcanis 535e955814 try to make auto archive upload 2021-04-05 00:37:03 +03:00
arcanis 0bd3ba626a implicit type conversion from command line 2021-04-04 23:53:30 +03:00
arcanis ffe6aec190 more options in setup command 2021-04-04 15:42:06 +03:00
arcanis 56c600e5ac fix check errors 2021-04-04 14:00:42 +03:00
arcanis 461883217d 100% coverage 2021-04-03 21:30:57 +03:00
arcanis 62d55eff19 add ability to fitler by dependency list 2021-04-02 04:20:39 +03:00
arcanis 534b5600b4 add ability to remove package from status page 2021-04-02 01:26:46 +03:00
arcanis 32cbafd12b Release 0.20.0 2021-04-01 02:38:59 +03:00
arcanis 880c70bd58 constistent classmethod and staticmethod usage
General idea is to use classmethod for every constructor and
statismethod otherwise.
Also use self and cls whenever it's possible to call static and class
methods
2021-03-31 04:29:08 +03:00
arcanis d449eb3c2e change arch specific section naming from section_arch to section:arch
Some archs can have _ in their name. Also in future we can use sections
with similar names
2021-03-31 02:31:14 +03:00
arcanis 17b5cd0751 Release 0.19.0 2021-03-31 02:19:44 +03:00
arcanis 2aef906fc8 add now argument to add command 2021-03-31 02:19:13 +03:00
arcanis e034327501 filter out every foreign arch in config 2021-03-31 01:55:39 +03:00
arcanis 5d79fcca22 read sign targets from correct path 2021-03-31 00:11:05 +03:00
arcanis 6e9dcca254 make configuration object arch-specific 2021-03-31 00:04:13 +03:00
arcanis fbf6748d4a more verbose variables 2021-03-30 05:29:13 +03:00
arcanis 2260e52d5c merge settings groups instead of using whole group 2021-03-30 04:58:15 +03:00
arcanis bd2b61494f move rsync and s3 options to configuration 2021-03-30 02:38:18 +03:00
arcanis 7280d30748 verbose help message 2021-03-30 02:25:23 +03:00
arcanis 710274065d add status badge 2021-03-30 02:01:59 +03:00
arcanis e0b09cefad rename gpg test to correct naming 2021-03-30 01:53:23 +03:00
arcanis 3b93510aad add more tests 2021-03-30 01:42:01 +03:00
arcanis 5003cabeb5 Release 0.18.0 2021-03-29 11:48:54 +03:00
arcanis bc6af9256b more properties to be shown in status pages 2021-03-29 11:48:32 +03:00
arcanis 1ac7c87317 architecture depending pacman.conf 2021-03-29 10:08:11 +03:00
arcanis 803b7bee1e add status update subcommand
also satisfy pylint with too big method with too much variables
2021-03-29 04:17:10 +03:00
arcanis 646190121a Release 0.17.0 2021-03-29 03:25:43 +03:00
arcanis 10e4f3b629 Setup command (#9)
* block issues without templates

* add setup subcommand

* handle devtools config correctly
2021-03-29 03:24:58 +03:00
arcanis 80a1f37c85 more templates 2021-03-29 00:13:20 +03:00
arcanis 751676a07e Add issue templates 2021-03-28 23:50:41 +03:00
arcanis e1a7071ce5 try to integrate with github workflows 2021-03-28 23:13:42 +03:00
arcanis 1605d185e2 remove unused import 2021-03-28 16:24:51 +03:00
arcanis 2fdf910e78 add sign command (#7) (#8) 2021-03-28 16:24:00 +03:00
arcanis 63dc43366b Release 0.16.0 2021-03-28 15:38:12 +03:00
arcanis 74a244f06c Add tests (#1) (#5)
* add models tests (#1)

also replace single quote to double one to confort PEP docstring
+ move _check_output to class properties to make it available for
mocking

* alpm tests implementation

* try to replace os with pathlib

* update tests for pathlib

* fix includes glob and trim version from dependencies

* build_tools package tests

* repository component tests

* add sign tests

* complete status tests

* handle exceptions in actual_version calls

* complete core tests

* move configuration to root conftest

* application tests

* complete application tests

* change copyright to more generic one

* base web tests

* complete web tests

* complete testkit

also add argument parsers test
2021-03-28 15:30:51 +03:00
1088 changed files with 26146 additions and 47442 deletions
+6
View File
@@ -0,0 +1,6 @@
skips:
- B101
- B104
- B105
- B106
- B404
-3
View File
@@ -12,6 +12,3 @@ __pycache__/
*.pyc
*.pyd
*.pyo
node_modules/
package-lock.json
+6 -18
View File
@@ -7,11 +7,6 @@ on:
tags:
- '*'
- '!*rc*'
workflow_dispatch:
permissions:
contents: read
packages: write
jobs:
docker-image:
@@ -22,22 +17,18 @@ jobs:
packages: write
steps:
- uses: docker/setup-qemu-action@v3
- uses: docker/setup-qemu-action@v2
- uses: docker/setup-buildx-action@v3
- name: Set image date
id: args
run: echo "::set-output name=date::$(date -d yesterday +'%Y-%m-%d')"
- uses: docker/setup-buildx-action@v2
- name: Login to docker hub
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to github container registry
uses: docker/login-action@v3
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
@@ -45,7 +36,7 @@ jobs:
- name: Extract docker metadata
id: meta
uses: docker/metadata-action@v5
uses: docker/metadata-action@v3
with:
images: |
arcan1s/ahriman
@@ -55,10 +46,7 @@ jobs:
type=edge
- name: Build an image and push
uses: docker/build-push-action@v6
uses: docker/build-push-action@v4
with:
build-args: |
BUILD_DATE=${{ steps.args.outputs.date }}
file: docker/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
-140
View File
@@ -1,140 +0,0 @@
name: Regress
on:
schedule:
- cron: 1 0 * * 0
workflow_dispatch:
permissions:
contents: read
jobs:
run-regress-tests:
runs-on: ubuntu-latest
services:
ahriman:
image: arcan1s/ahriman:edge
env:
AHRIMAN_PORT: 8080
AHRIMAN_UNIX_SOCKET: /var/lib/ahriman/ahriman/ahriman-web.sock
options: --privileged --entrypoint entrypoint-web
ports:
- 8080
volumes:
- repo:/var/lib/ahriman
container:
image: arcan1s/ahriman:edge
env:
AHRIMAN_DEBUG: y
AHRIMAN_OUTPUT: console
AHRIMAN_PORT: 8080
AHRIMAN_UNIX_SOCKET: /var/lib/ahriman/ahriman/ahriman-web.sock
options: --privileged
volumes:
- repo:/var/lib/ahriman
steps:
- run: pacman -Sy
- name: Init repository
run: entrypoint help
- name: Print configuration
run: |
ahriman \
--log-handler "$AHRIMAN_OUTPUT" \
service-config
- name: Validate configuration
run: |
ahriman \
--log-handler "$AHRIMAN_OUTPUT" \
service-config-validate
- name: Create a user
run: |
sudo -u ahriman ahriman \
--log-handler "$AHRIMAN_OUTPUT" \
user-add \
--packager "github actions <actions@github.com>" \
--password ahriman \
--role full \
ahriman
- name: Fetch users
run: |
ahriman \
--log-handler "$AHRIMAN_OUTPUT" \
user-list \
--exit-code
- name: Add package
run: |
sudo -u ahriman ahriman \
--log-handler "$AHRIMAN_OUTPUT" \
package-add \
--exit-code \
--now \
--refresh \
ahriman
- name: Update status of the package
run: |
ahriman \
--log-handler "$AHRIMAN_OUTPUT" \
package-status-update \
--status failed \
ahriman
- name: Request status of the package
run: |
ahriman \
--log-handler "$AHRIMAN_OUTPUT" \
package-status \
--exit-code \
--info \
--status failed \
ahriman
- name: Update packages
run: |
sudo -u ahriman ahriman \
--log-handler "$AHRIMAN_OUTPUT" \
repo-update \
--exit-code \
|| true
- name: Add patch
run: |
echo '${pkgver%%.*}' | \
sudo -u ahriman ahriman \
--log-handler "$AHRIMAN_OUTPUT" \
patch-add \
ahriman \
pkgrel
- name: Retrieve patches
run: |
ahriman \
--log-handler "$AHRIMAN_OUTPUT" \
patch-list \
--exit-code \
ahriman
- name: Rebuild packages
run: |
sudo -u ahriman ahriman \
--log-handler "$AHRIMAN_OUTPUT" \
repo-rebuild \
--depends-on python \
--exit-code
- name: Remove package
run: |
sudo -u ahriman ahriman \
--log-handler "$AHRIMAN_OUTPUT" \
package-remove \
ahriman
+7 -13
View File
@@ -5,24 +5,13 @@ on:
tags:
- '*'
permissions:
contents: write
jobs:
make-release:
runs-on: ubuntu-latest
container:
image: archlinux:base
options: -w /build
volumes:
- ${{ github.workspace }}:/build
steps:
- run: pacman --noconfirm -Syu base-devel git python-tox
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: Extract version
id: version
@@ -35,13 +24,18 @@ jobs:
token: ${{ secrets.GITHUB_TOKEN }}
filter: 'Release \d+\.\d+\.\d+'
- name: Install dependencies
uses: ConorMacBride/install-package@v1.1.0
with:
apt: tox
- name: Create archive
run: tox -e archive
env:
VERSION: ${{ steps.version.outputs.VERSION }}
- name: Publish release
uses: softprops/action-gh-release@v2
uses: softprops/action-gh-release@v1
with:
body: |
${{ steps.changelog.outputs.compareurl }}
+12 -15
View File
@@ -8,17 +8,19 @@ set -ex
# install dependencies
echo -e '[arcanisrepo]\nServer = https://repo.arcanis.me/$arch\nSigLevel = Never' | tee -a /etc/pacman.conf
# refresh the image
pacman -Syyu --noconfirm
pacman -Syu --noconfirm
# main dependencies
pacman -S --noconfirm devtools git npm pyalpm python-bcrypt python-filelock python-inflection python-pyelftools python-requests python-systemd sudo
pacman -Sy --noconfirm devtools git pyalpm python-cerberus python-inflection python-passlib python-requests python-srcinfo python-systemd sudo
# make dependencies
pacman -S --noconfirm --asdeps base-devel python-build python-hatchling python-installer python-tox python-wheel
pacman -Sy --noconfirm --asdeps base-devel python-build python-flit python-installer python-tox python-wheel
# optional dependencies
if [[ -z $MINIMAL_INSTALL ]]; then
# VCS support
pacman -Sy --noconfirm breezy darcs mercurial subversion
# web server
pacman -S --noconfirm python-aioauth-client python-aiohttp python-aiohttp-apispec-git python-aiohttp-cors python-aiohttp-jinja2 python-aiohttp-security python-aiohttp-session python-aiohttp-sse-git python-cryptography python-jinja
pacman -Sy --noconfirm python-aioauth-client python-aiohttp python-aiohttp-apispec-git python-aiohttp-cors python-aiohttp-jinja2 python-aiohttp-security python-aiohttp-session python-cryptography python-jinja
# additional features
pacman -S --noconfirm gnupg ipython python-boto3 python-cerberus python-matplotlib rsync
pacman -Sy --noconfirm gnupg python-boto3 rsync
fi
# FIXME since 1.0.4 devtools requires dbus to be run, which doesn't work now in container
cp "docker/systemd-nspawn.sh" "/usr/local/bin/systemd-nspawn"
@@ -26,16 +28,11 @@ cp "docker/systemd-nspawn.sh" "/usr/local/bin/systemd-nspawn"
# create fresh tarball
tox -e archive
# run makepkg
PKGVER=$(python -c "from src.ahriman import __version__; print(__version__)")
mv "dist/ahriman-$PKGVER.tar.gz" package/archlinux
mv dist/ahriman-*.tar.gz package/archlinux
chmod +777 package/archlinux # because fuck you that's why
cd package/archlinux
sudo -u nobody -- makepkg -cf --skipchecksums --noconfirm
sudo -u nobody -- makepkg --packagelist | grep "ahriman-core-$PKGVER" | pacman -U --noconfirm --nodeps -
if [[ -z $MINIMAL_INSTALL ]]; then
sudo -u nobody -- makepkg --packagelist | grep "ahriman-triggers-$PKGVER" | pacman -U --noconfirm --nodeps -
sudo -u nobody -- makepkg --packagelist | grep "ahriman-web-$PKGVER" | pacman -U --noconfirm --nodeps -
fi
sudo -u nobody -- makepkg --packagelist | grep -v -- -debug- | pacman -U --noconfirm -
# create machine-id which is required by build tools
systemd-machine-id-setup
@@ -45,13 +42,13 @@ pacman -Qdtq | pacman -Rscn --noconfirm -
# initial setup command as root
[[ -z $MINIMAL_INSTALL ]] && WEB_ARGS=("--web-port" "8080")
ahriman -a x86_64 -r "github" service-setup --packager "ahriman bot <ahriman@example.com>" "${WEB_ARGS[@]}"
# validate configuration
ahriman service-config-validate --exit-code
# enable services
systemctl enable ahriman-web
systemctl enable ahriman@x86_64-github.timer
if [[ -z $MINIMAL_INSTALL ]]; then
# validate configuration
ahriman service-config-validate --exit-code
# run web service (detached)
systemctl enable ahriman-web
sudo -u ahriman -- ahriman web &
WEB_PID=$!
fi
+3 -7
View File
@@ -7,10 +7,6 @@ on:
pull_request:
branches:
- master
workflow_dispatch:
permissions:
contents: read
jobs:
run-setup-minimal:
@@ -19,12 +15,12 @@ jobs:
container:
image: archlinux:base
options: --privileged -w /build
volumes:
- ${{ github.workspace }}:/build
options: --privileged -w /build
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: Setup the minimal service in arch linux container
run: .github/workflows/setup.sh minimal
@@ -40,7 +36,7 @@ jobs:
options: --privileged -w /build
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: Setup the service in arch linux container
run: .github/workflows/setup.sh
+10
View File
@@ -0,0 +1,10 @@
#!/bin/bash
# Install dependencies and run test in container
set -ex
# install dependencies
pacman --noconfirm -Syu base-devel python-tox
# run test and check targets
tox
+4 -19
View File
@@ -7,12 +7,6 @@ on:
pull_request:
branches:
- master
schedule:
- cron: 1 0 * * *
workflow_dispatch:
permissions:
contents: read
jobs:
run-tests:
@@ -21,21 +15,12 @@ jobs:
container:
image: archlinux:base
options: -w /build
volumes:
- ${{ github.workspace }}:/build
options: -w /build
steps:
- run: pacman --noconfirm -Syu base-devel git npm python-tox
- uses: actions/checkout@v3
- run: git config --global --add safe.directory *
- uses: actions/checkout@v4
- name: Run check and tests
run: tox
- name: Generate documentation and check if there are untracked changes
run: |
tox -e docs
[ -z "$(git status --porcelain docs/*.rst)" ]
- name: Run check and tests in arch linux container
run: .github/workflows/tests.sh
-9
View File
@@ -99,12 +99,3 @@ status_cache.json
*.db
docs/html/
# Frontend
node_modules/
package-lock.json
ahriman-web/package/share/ahriman/templates/static/index.js
ahriman-web/package/share/ahriman/templates/static/index.css
# local configs
/*.ini
-45
View File
@@ -1,45 +0,0 @@
[tool.pylint.main]
init-hook = "sys.path.append('tools')"
load-plugins = [
"pylint.extensions.docparams",
"pylint.extensions.bad_builtin",
"pylint_plugins.definition_order",
"pylint_plugins.import_order",
]
[tool.pylint.classes]
bad-functions = [
"print",
]
[tool.pylint.design]
max-parents = 15
[tool.pylint."messages control"]
disable = [
"raw-checker-failed",
"bad-inline-option",
"locally-disabled",
"file-ignored",
"suppressed-message",
"useless-suppression",
"deprecated-pragma",
"use-symbolic-message-instead",
"use-implicit-booleaness-not-comparison-to-string",
"use-implicit-booleaness-not-comparison-to-zero",
"missing-module-docstring",
"line-too-long",
"no-name-in-module",
"import-outside-toplevel",
"invalid-name",
"raise-missing-from",
"wrong-import-order",
"too-few-public-methods",
"too-many-instance-attributes",
"broad-exception-caught",
"fixme",
"too-many-arguments",
"duplicate-code",
"cyclic-import",
"too-many-positional-arguments",
]
+647
View File
@@ -0,0 +1,647 @@
[MAIN]
# Analyse import fallback blocks. This can be used to support both Python 2 and
# 3 compatible code, which means that the block might have code that exists
# only in one or another interpreter, leading to false positives when analysed.
analyse-fallback-blocks=no
# Clear in-memory caches upon conclusion of linting. Useful if running pylint
# in a server-like mode.
clear-cache-post-run=no
# Load and enable all available extensions. Use --list-extensions to see a list
# all available extensions.
#enable-all-extensions=
# In error mode, messages with a category besides ERROR or FATAL are
# suppressed, and no reports are done by default. Error mode is compatible with
# disabling specific errors.
#errors-only=
# Always return a 0 (non-error) status code, even if lint errors are found.
# This is primarily useful in continuous integration scripts.
#exit-zero=
# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code.
extension-pkg-allow-list=
# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code. (This is an alternative name to extension-pkg-allow-list
# for backward compatibility.)
extension-pkg-whitelist=
# Return non-zero exit code if any of these messages/categories are detected,
# even if score is above --fail-under value. Syntax same as enable. Messages
# specified are enabled, while categories only check already-enabled messages.
fail-on=
# Specify a score threshold under which the program will exit with error.
fail-under=10
# Interpret the stdin as a python script, whose filename needs to be passed as
# the module_or_package argument.
#from-stdin=
# Files or directories to be skipped. They should be base names, not paths.
ignore=CVS
# Add files or directories matching the regular expressions patterns to the
# ignore-list. The regex matches against paths and can be in Posix or Windows
# format. Because '\\' represents the directory delimiter on Windows systems,
# it can't be used as an escape character.
ignore-paths=
# Files or directories matching the regular expression patterns are skipped.
# The regex matches against base names, not paths. The default value ignores
# Emacs file locks
ignore-patterns=^\.#
# List of module names for which member attributes should not be checked
# (useful for modules/projects where namespaces are manipulated during runtime
# and thus existing member attributes cannot be deduced by static analysis). It
# supports qualified module names, as well as Unix pattern matching.
ignored-modules=
# Python code to execute, usually for sys.path manipulation such as
# pygtk.require().
init-hook='import sys; sys.path.append("pylint_plugins")'
# Use multiple processes to speed up Pylint. Specifying 0 will auto-detect the
# number of processors available to use, and will cap the count on Windows to
# avoid hangs.
jobs=1
# Control the amount of potential inferred values when inferring a single
# object. This can help the performance when dealing with large functions or
# complex, nested conditions.
limit-inference-results=100
# List of plugins (as comma separated values of python module names) to load,
# usually to register additional checkers.
load-plugins=pylint.extensions.docparams,
definition_order,
import_order,
# Pickle collected data for later comparisons.
persistent=yes
# Minimum Python version to use for version dependent checks. Will default to
# the version used to run pylint.
py-version=3.11
# Discover python modules and packages in the file system subtree.
recursive=no
# Add paths to the list of the source roots. Supports globbing patterns. The
# source root is an absolute path or a path relative to the current working
# directory used to determine a package namespace for modules located under the
# source root.
source-roots=
# When enabled, pylint would attempt to guess common misconfiguration and emit
# user-friendly hints instead of false-positive error messages.
suggestion-mode=yes
# Allow loading of arbitrary C extensions. Extensions are imported into the
# active Python interpreter and may run arbitrary code.
unsafe-load-any-extension=no
# In verbose mode, extra non-checker-related info will be displayed.
#verbose=
[BASIC]
# Naming style matching correct argument names.
argument-naming-style=snake_case
# Regular expression matching correct argument names. Overrides argument-
# naming-style. If left empty, argument names will be checked with the set
# naming style.
#argument-rgx=
# Naming style matching correct attribute names.
attr-naming-style=snake_case
# Regular expression matching correct attribute names. Overrides attr-naming-
# style. If left empty, attribute names will be checked with the set naming
# style.
#attr-rgx=
# Bad variable names which should always be refused, separated by a comma.
bad-names=foo,
bar,
baz,
toto,
tutu,
tata
# Bad variable names regexes, separated by a comma. If names match any regex,
# they will always be refused
bad-names-rgxs=
# Naming style matching correct class attribute names.
class-attribute-naming-style=any
# Regular expression matching correct class attribute names. Overrides class-
# attribute-naming-style. If left empty, class attribute names will be checked
# with the set naming style.
#class-attribute-rgx=
# Naming style matching correct class constant names.
class-const-naming-style=UPPER_CASE
# Regular expression matching correct class constant names. Overrides class-
# const-naming-style. If left empty, class constant names will be checked with
# the set naming style.
#class-const-rgx=
# Naming style matching correct class names.
class-naming-style=PascalCase
# Regular expression matching correct class names. Overrides class-naming-
# style. If left empty, class names will be checked with the set naming style.
#class-rgx=
# Naming style matching correct constant names.
const-naming-style=UPPER_CASE
# Regular expression matching correct constant names. Overrides const-naming-
# style. If left empty, constant names will be checked with the set naming
# style.
#const-rgx=
# Minimum line length for functions/classes that require docstrings, shorter
# ones are exempt.
docstring-min-length=-1
# Naming style matching correct function names.
function-naming-style=snake_case
# Regular expression matching correct function names. Overrides function-
# naming-style. If left empty, function names will be checked with the set
# naming style.
#function-rgx=
# Good variable names which should always be accepted, separated by a comma.
good-names=i,
j,
k,
ex,
Run,
_
# Good variable names regexes, separated by a comma. If names match any regex,
# they will always be accepted
good-names-rgxs=
# Include a hint for the correct naming format with invalid-name.
include-naming-hint=no
# Naming style matching correct inline iteration names.
inlinevar-naming-style=any
# Regular expression matching correct inline iteration names. Overrides
# inlinevar-naming-style. If left empty, inline iteration names will be checked
# with the set naming style.
#inlinevar-rgx=
# Naming style matching correct method names.
method-naming-style=snake_case
# Regular expression matching correct method names. Overrides method-naming-
# style. If left empty, method names will be checked with the set naming style.
#method-rgx=
# Naming style matching correct module names.
module-naming-style=snake_case
# Regular expression matching correct module names. Overrides module-naming-
# style. If left empty, module names will be checked with the set naming style.
#module-rgx=
# Colon-delimited sets of names that determine each other's naming style when
# the name regexes allow several styles.
name-group=
# Regular expression which should only match function or class names that do
# not require a docstring.
no-docstring-rgx=
# List of decorators that produce properties, such as abc.abstractproperty. Add
# to this list to register other decorators that produce valid properties.
# These decorators are taken in consideration only for invalid-name.
property-classes=abc.abstractproperty
# Regular expression matching correct type alias names. If left empty, type
# alias names will be checked with the set naming style.
#typealias-rgx=
# Regular expression matching correct type variable names. If left empty, type
# variable names will be checked with the set naming style.
#typevar-rgx=
# Naming style matching correct variable names.
variable-naming-style=snake_case
# Regular expression matching correct variable names. Overrides variable-
# naming-style. If left empty, variable names will be checked with the set
# naming style.
#variable-rgx=
[CLASSES]
# Warn about protected attribute access inside special methods
check-protected-access-in-special-methods=no
# List of method names used to declare (i.e. assign) instance attributes.
defining-attr-methods=__init__,
__new__,
setUp,
asyncSetUp,
__post_init__
# List of member names, which should be excluded from the protected access
# warning.
exclude-protected=_asdict,_fields,_replace,_source,_make,os._exit
# List of valid names for the first argument in a class method.
valid-classmethod-first-arg=cls
# List of valid names for the first argument in a metaclass class method.
valid-metaclass-classmethod-first-arg=mcs
[DESIGN]
# List of regular expressions of class ancestor names to ignore when counting
# public methods (see R0903)
exclude-too-few-public-methods=
# List of qualified class names to ignore when counting class parents (see
# R0901)
ignored-parents=
# Maximum number of arguments for function / method.
max-args=5
# Maximum number of attributes for a class (see R0902).
max-attributes=7
# Maximum number of boolean expressions in an if statement (see R0916).
max-bool-expr=5
# Maximum number of branch for function / method body.
max-branches=12
# Maximum number of locals for function / method body.
max-locals=15
# Maximum number of parents for a class (see R0901).
max-parents=7
# Maximum number of public methods for a class (see R0904).
max-public-methods=20
# Maximum number of return / yield for function / method body.
max-returns=6
# Maximum number of statements in function / method body.
max-statements=50
# Minimum number of public methods for a class (see R0903).
min-public-methods=2
[EXCEPTIONS]
# Exceptions that will emit a warning when caught.
overgeneral-exceptions=builtins.BaseException,builtins.Exception
[FORMAT]
# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
expected-line-ending-format=
# Regexp for a line that is allowed to be longer than the limit.
ignore-long-lines=^\s*(# )?<?https?://\S+>?$
# Number of spaces of indent required inside a hanging or continued line.
indent-after-paren=4
# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
# tab).
indent-string=' '
# Maximum number of characters on a single line.
max-line-length=100
# Maximum number of lines in a module.
max-module-lines=1000
# Allow the body of a class to be on the same line as the declaration if body
# contains single statement.
single-line-class-stmt=no
# Allow the body of an if to be on the same line as the test if there is no
# else.
single-line-if-stmt=no
[IMPORTS]
# List of modules that can be imported at any level, not just the top level
# one.
allow-any-import-level=
# Allow explicit reexports by alias from a package __init__.
allow-reexport-from-package=no
# Allow wildcard imports from modules that define __all__.
allow-wildcard-with-all=no
# Deprecated modules which should not be used, separated by a comma.
deprecated-modules=
# Output a graph (.gv or any supported image format) of external dependencies
# to the given file (report RP0402 must not be disabled).
ext-import-graph=
# Output a graph (.gv or any supported image format) of all (i.e. internal and
# external) dependencies to the given file (report RP0402 must not be
# disabled).
import-graph=
# Output a graph (.gv or any supported image format) of internal dependencies
# to the given file (report RP0402 must not be disabled).
int-import-graph=
# Force import order to recognize a module as part of the standard
# compatibility libraries.
known-standard-library=
# Force import order to recognize a module as part of a third party library.
known-third-party=enchant
# Couples of modules and preferred modules, separated by a comma.
preferred-modules=
[LOGGING]
# The type of string formatting that logging methods do. `old` means using %
# formatting, `new` is for `{}` formatting.
logging-format-style=old
# Logging modules to check that the string format arguments are in logging
# function parameter format.
logging-modules=logging
[MESSAGES CONTROL]
# Only show warnings with the listed confidence levels. Leave empty to show
# all. Valid levels: HIGH, CONTROL_FLOW, INFERENCE, INFERENCE_FAILURE,
# UNDEFINED.
confidence=HIGH,
CONTROL_FLOW,
INFERENCE,
INFERENCE_FAILURE,
UNDEFINED
# Disable the message, report, category or checker with the given id(s). You
# can either give multiple identifiers separated by comma (,) or put this
# option multiple times (only on the command line, not in the configuration
# file where it should appear only once). You can also use "--disable=all" to
# disable everything first and then re-enable specific checks. For example, if
# you want to run only the similarities checker, you can use "--disable=all
# --enable=similarities". If you want to run only the classes checker, but have
# no Warning level messages displayed, use "--disable=all --enable=classes
# --disable=W".
disable=raw-checker-failed,
bad-inline-option,
locally-disabled,
file-ignored,
suppressed-message,
useless-suppression,
deprecated-pragma,
use-symbolic-message-instead,
missing-module-docstring,
line-too-long,
no-name-in-module,
import-outside-toplevel,
invalid-name,
raise-missing-from,
wrong-import-order,
too-few-public-methods,
too-many-instance-attributes,
broad-except,
fixme,
too-many-arguments,
duplicate-code,
cyclic-import,
# Enable the message, report, category or checker with the given id(s). You can
# either give multiple identifier separated by comma (,) or put this option
# multiple time (only on the command line, not in the configuration file where
# it should appear only once). See also the "--disable" option for examples.
enable=c-extension-no-member
[METHOD_ARGS]
# List of qualified names (i.e., library.method) which require a timeout
# parameter e.g. 'requests.api.get,requests.api.post'
timeout-methods=requests.api.delete,requests.api.get,requests.api.head,requests.api.options,requests.api.patch,requests.api.post,requests.api.put,requests.api.request
[MISCELLANEOUS]
# List of note tags to take in consideration, separated by a comma.
notes=FIXME,
XXX,
TODO
# Regular expression of note tags to take in consideration.
notes-rgx=
[REFACTORING]
# Maximum number of nested blocks for function / method body
max-nested-blocks=5
# Complete name of functions that never returns. When checking for
# inconsistent-return-statements if a never returning function is called then
# it will be considered as an explicit return statement and no message will be
# printed.
never-returning-functions=sys.exit,argparse.parse_error
[REPORTS]
# Python expression which should return a score less than or equal to 10. You
# have access to the variables 'fatal', 'error', 'warning', 'refactor',
# 'convention', and 'info' which contain the number of messages in each
# category, as well as 'statement' which is the total number of statements
# analyzed. This score is used by the global evaluation report (RP0004).
evaluation=max(0, 0 if fatal else 10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10))
# Template used to display messages. This is a python new-style format string
# used to format the message information. See doc for all details.
msg-template=
# Set the output format. Available formats are text, parseable, colorized, json
# and msvs (visual studio). You can also give a reporter class, e.g.
# mypackage.mymodule.MyReporterClass.
#output-format=
# Tells whether to display a full report or only the messages.
reports=no
# Activate the evaluation score.
score=yes
[SIMILARITIES]
# Comments are removed from the similarity computation
ignore-comments=yes
# Docstrings are removed from the similarity computation
ignore-docstrings=yes
# Imports are removed from the similarity computation
ignore-imports=yes
# Signatures are removed from the similarity computation
ignore-signatures=yes
# Minimum lines number of a similarity.
min-similarity-lines=4
[SPELLING]
# Limits count of emitted suggestions for spelling mistakes.
max-spelling-suggestions=4
# Spelling dictionary name. No available dictionaries : You need to install
# both the python package and the system dependency for enchant to work..
spelling-dict=
# List of comma separated words that should be considered directives if they
# appear at the beginning of a comment and should not be checked.
spelling-ignore-comment-directives=fmt: on,fmt: off,noqa:,noqa,nosec,isort:skip,mypy:
# List of comma separated words that should not be checked.
spelling-ignore-words=
# A path to a file that contains the private dictionary; one word per line.
spelling-private-dict-file=
# Tells whether to store unknown words to the private dictionary (see the
# --spelling-private-dict-file option) instead of raising a message.
spelling-store-unknown-words=no
[STRING]
# This flag controls whether inconsistent-quotes generates a warning when the
# character used as a quote delimiter is used inconsistently within a module.
check-quote-consistency=no
# This flag controls whether the implicit-str-concat should generate a warning
# on implicit string concatenation in sequences defined over several lines.
check-str-concat-over-line-jumps=no
[TYPECHECK]
# List of decorators that produce context managers, such as
# contextlib.contextmanager. Add to this list to register other decorators that
# produce valid context managers.
contextmanager-decorators=contextlib.contextmanager
# List of members which are set dynamically and missed by pylint inference
# system, and so shouldn't trigger E1101 when accessed. Python regular
# expressions are accepted.
generated-members=
# Tells whether to warn about missing members when the owner of the attribute
# is inferred to be None.
ignore-none=yes
# This flag controls whether pylint should warn about no-member and similar
# checks whenever an opaque object is returned when inferring. The inference
# can return multiple potential results while evaluating a Python object, but
# some branches might not be evaluated, which results in partial inference. In
# that case, it might be useful to still emit no-member and other checks for
# the rest of the inferred objects.
ignore-on-opaque-inference=yes
# List of symbolic message names to ignore for Mixin members.
ignored-checks-for-mixins=no-member,
not-async-context-manager,
not-context-manager,
attribute-defined-outside-init
# List of class names for which member attributes should not be checked (useful
# for classes with dynamically set attributes). This supports the use of
# qualified names.
ignored-classes=optparse.Values,thread._local,_thread._local,argparse.Namespace
# Show a hint with possible names when a member name was not found. The aspect
# of finding the hint is based on edit distance.
missing-member-hint=yes
# The minimum edit distance a name should have in order to be considered a
# similar match for a missing member name.
missing-member-hint-distance=1
# The total number of similar names that should be taken in consideration when
# showing a hint for a missing member.
missing-member-max-choices=1
# Regex pattern to define which classes are considered mixins.
mixin-class-rgx=.*[Mm]ixin
# List of decorators that change the signature of a decorated function.
signature-mutators=
[VARIABLES]
# List of additional names supposed to be defined in builtins. Remember that
# you should avoid defining new builtins when possible.
additional-builtins=
# Tells whether unused global variables should be treated as a violation.
allow-global-unused-variables=yes
# List of names allowed to shadow builtins
allowed-redefined-builtins=
# List of strings which can identify a callback function by name. A callback
# name must start or end with one of those strings.
callbacks=cb_,
_cb
# A regular expression matching the name of dummy variables (i.e. expected to
# not be used).
dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_
# Argument names that match this expression will be ignored.
ignored-argument-names=_.*|^ignored_|^unused_
# Tells whether we should check for unused import in __init__ files.
init-import=no
# List of qualified module names which can have objects that can redefine
# builtins.
redefining-builtins-modules=six.moves,past.builtins,future.builtins,builtins,io
-7
View File
@@ -1,7 +0,0 @@
[pytest]
addopts = --cov=ahriman --cov-report=term-missing:skip-covered --no-cov-on-fail --cov-fail-under=100 --spec
asyncio_default_fixture_loop_scope = function
asyncio_mode = auto
pythonpath = tests
resource-path.directory-name-test-resources = ../../tests/testresources
spec_test_format = {result} {docstring_summary}
+8 -5
View File
@@ -1,15 +1,18 @@
version: 2
build:
os: ubuntu-lts-latest
os: ubuntu-20.04
tools:
python: "3.13"
apt_packages:
- graphviz
python: "3.12"
python:
install:
- requirements: docs/requirements.txt
- method: pip
path: .
extra_requirements:
- docs
- s3
- web
formats:
- pdf
+28 -65
View File
@@ -36,7 +36,6 @@ Again, the most checks can be performed by `tox` command, though some additional
Notes:
Very important note about this function
Probably multi-line
Args:
argument(str): an argument. This argument has
@@ -71,7 +70,6 @@ Again, the most checks can be performed by `tox` command, though some additional
Attributes:
CLAZZ_ATTRIBUTE(int): (class attribute) a brand-new class attribute
instance_attribute(str): an instance attribute
with the long description
Examples:
Very informative class usage example, e.g.::
@@ -80,10 +78,12 @@ Again, the most checks can be performed by `tox` command, though some additional
>>> clazz = Clazz()
"""
CLAZZ_ATTRIBUTE: ClassVar[int] = 42
CLAZZ_ATTRIBUTE = 42
def __init__(self, *args: Any, **kwargs: Any) -> None:
"""
default constructor
Args:
*args(Any): positional arguments
**kwargs(Any): keyword arguments
@@ -91,12 +91,9 @@ Again, the most checks can be performed by `tox` command, though some additional
self.instance_attribute = ""
```
Note missing comment for the `__init__` method, which is the special case.
* Type annotations are the must, even for local functions. For the function argument `self` (for instance methods) and `cls` (for class methods) should not be annotated.
* For collection types built-in classes must be used if possible (e.g. `dict` instead of `typing.Dict`, `tuple` instead of `typing.Tuple`). In case if built-in type is not available, but `collections.abc` provides interface, it must be used (e.g. `collections.abc.Awaitable` instead of `typing.Awaitable`, `collections.abc.Iterable` instead of `typing.Iterable`). For union classes, the bar operator (`|`) must be used (e.g. `float | int` instead of `typing.Union[float, int]`), which also includes `typing.Optional` (e.g. `str | None` instead of `Optional[str]`).
* For collection types built-in classes must be used if possible (e.g. `dict` instead of `typing.Dict`, `tuple` instead of `typing.Tuple`). In case if built-in type is not available, but `collections.abc` provides interface, it must be used (e.g. `collections.abc.Awaitable` instead of `typing.Awaitable`, `collections.abc.Iterable` instead of `typing.Iterable`). For union classes, the bar operator (`|`) must be used (e.g. `float | int` instead of `typing.Union[float, int]`), which also includes `typinng.Optional` (e.g. `str | None` instead of `Optional[str]`).
* `classmethod` should (almost) always return `Self`. In case of mypy warning (e.g. if there is a branch in which function doesn't return the instance of `cls`) consider using `staticmethod` instead.
* Class attributes must be decorated as `ClassVar[...]`.
* Recommended order of function definitions in class:
```python
@@ -125,7 +122,7 @@ Again, the most checks can be performed by `tox` command, though some additional
def __hash__(self) -> int: ... # basically any magic (or look-alike) method
```
Methods inside one group should be ordered alphabetically, the only exceptions are `__init__` (`__post_init__` for dataclasses), `__new__` and `__del__` methods which should be defined first. For test methods it is recommended to follow the order in which functions are defined. Same idea applies to frontend classes.
Methods inside one group should be ordered alphabetically, the only exceptions are `__init__` (`__post_init__` for dataclasses), `__new__` and `__del__` methods which should be defined first. For test methods it is recommended to follow the order in which functions are defined.
Though, we would like to highlight abstract methods (i.e. ones which raise `NotImplementedError`), we still keep in global order at the moment.
@@ -135,7 +132,7 @@ Again, the most checks can be performed by `tox` command, though some additional
* For any path interactions `pathlib.Path` must be used.
* Configuration interactions must go through `ahriman.core.configuration.Configuration` class instance.
* In case if class load requires some actions, it is recommended to create class method which can be used for class instantiating.
* The most (expected) exceptions must be handled and printed to log, allowing service to continue work. However, fatal and (in some cases) unexpected exceptions may lead to the application termination.
* The code must follow the exception safety, unless it is explicitly asked by end user. It means that most exceptions must be handled and printed to log, no other actions must be done (e.g. raising another exception).
* Exceptions without parameters should be raised without parentheses, e.g.:
```python
@@ -165,16 +162,10 @@ Again, the most checks can be performed by `tox` command, though some additional
# Blank line again and package imports
from ahriman.core.configuration import Configuration
# Multiline import example
from ahriman.core.database.operations import (
AuthOperations,
BuildOperations,
)
```
* Imports goes in alphabetical order, no relative imports allowed. Same rule applies to frontend classes.
* One file should define only one class, exception is class satellites in case if file length remains less than 400 lines. Same rule applies to frontend classes.
* It is possible to create file which contains some functions (e.g. `ahriman.core.utils`), but in this case you would need to define `__all__` attribute.
* One file should define only one class, exception is class satellites in case if file length remains less than 400 lines.
* It is possible to create file which contains some functions (e.g. `ahriman.core.util`), but in this case you would need to define `__all__` attribute.
* The file size mentioned above must be applicable in general. In case of big classes consider splitting them into traits. Note, however, that `pylint` includes comments and docstrings into counter, thus you need to check file size by other tools.
* No global variable is allowed outside of `ahriman` module. `ahriman.core.context` is also special case.
* Single quotes are not allowed. The reason behind this restriction is the fact that docstrings must be written by using double quotes only, and we would like to make style consistent.
@@ -182,10 +173,11 @@ Again, the most checks can be performed by `tox` command, though some additional
* Web API methods must be documented by using `aiohttp_apispec` library. The schema testing mostly should be implemented in related view class tests. Recommended example for documentation (excluding comments):
```python
import aiohttp_apispec
from marshmallow import Schema, fields
from ahriman.web.apispec.decorators import apidocs
from ahriman.web.schemas import PackageNameSchema, PaginationSchema
from ahriman.web.schemas import AuthSchema, ErrorSchema, PackageNameSchema, PaginationSchema
from ahriman.web.views.base import BaseView
@@ -204,31 +196,36 @@ Again, the most checks can be performed by `tox` command, though some additional
POST_PERMISSION = ...
ROUTES = ...
@apidocs(
@aiohttp_apispec.docs(
tags=["Tag"],
summary="Do foo",
description="Extended description of the method which does foo",
error_400_enabled=True, # exception raised by this method
error_404_description="Repository is unknown",
schema=ResponseSchema, # leave empty if no responses here
match_schema=PackageNameSchema,
query_schema=PaginationSchema,
body_schema=RequestSchema(many=True),
responses={
200: {"description": "Success response", "schema": ResponseSchema},
204: {"description": "Success response"}, # example without json schema response
400: {"description": "Bad data is supplied", "schema": ErrorSchema}, # exception raised by this method
401: {"description": "Authorization required", "schema": ErrorSchema}, # should be always presented
403: {"description": "Access is forbidden", "schema": ErrorSchema}, # should be always presented
404: {"description": "Repository is unknown", "schema": ErrorSchema}, # include if BaseView.service() method is called
500: {"description": "Internal server error", "schema": ErrorSchema}, # should be always presented
},
security=[{"token": [POST_PERMISSION]}],
)
@aiohttp_apispec.cookies_schema(AuthSchema) # should be always presented
@aiohttp_apispec.match_info_schema(PackageNameSchema)
@aiohttp_apispec.querystring_schema(PaginationSchema)
@aiohttp_apispec.json_schema(RequestSchema(many=True))
async def post(self) -> None: ...
```
* It is allowed to change web API to add new fields or remove optional ones. However, in case of model changes, new API version must be introduced.
* On the other hand, it is allowed to change method signatures, however, it is recommended to add new parameters as optional if possible. Deprecated API can be dropped during major release.
* Enumerations (`Enum` classes) are allowed and recommended. However, it is recommended to use `StrEnum` class if there are from/to string conversions and `IntEnum` otherwise.
* `Generator` return type is not allowed. Generator functions must return generic `Iterator` object. Documentation should be described as `Yields`, however, because of pylint checks. Unfortunately, `Iterable` return type is not available for generators also, because of specific `contextlib.contextmanager` case.
### Other checks
The projects also uses typing checks (provided by `mypy`) and some linter checks provided by `pylint` and `bandit`. Those checks must be passed successfully for any open pull requests.
Frontend checks normally are performed by `eslint` (e.g. `npx run eslint`).
## Developers how to
### Run automated checks
@@ -237,50 +234,16 @@ Frontend checks normally are performed by `eslint` (e.g. `npx run eslint`).
tox
```
Must be usually done before any pushes.
### Generate documentation templates
```shell
tox -e docs
```
Must be usually done if there are changes in modules structure.
### Create release
```shell
tox -m release -- major.minor.patch
tox -m release -- x.y.z
```
The command above will generate documentation, tags, etc., and will push them to GitHub. Other things will be handled by GitHub workflows automatically.
### Hotfixes policy
Sometimes it is required to publish hotfix with specific commits, but some features have been already committed, which should not be included to the hotfix. In this case, some manual steps are required:
1. Create new branch from the last stable release (`major.minor.patch`):
```shell
git checkout -b release/major.minor major.minor.patch
```
2. Cherry-pick desired commit(s):
```shell
git cherry-pick <commit-sha>
```
Alternatively, make changes to the new branch and commit them.
3. Push newly created branch to remote:
```shell
git push --set-upstream origin release/major.minor
```
4. Proceed to release as usual:
```shell
tox -m release -- major.minor.patch+1
```
+65
View File
@@ -0,0 +1,65 @@
FROM archlinux:base
# image configuration
ENV AHRIMAN_ARCHITECTURE="x86_64"
ENV AHRIMAN_DEBUG=""
ENV AHRIMAN_FORCE_ROOT=""
ENV AHRIMAN_HOST="0.0.0.0"
ENV AHRIMAN_MULTILIB="yes"
ENV AHRIMAN_OUTPUT=""
ENV AHRIMAN_PACKAGER="ahriman bot <ahriman@example.com>"
ENV AHRIMAN_PACMAN_MIRROR=""
ENV AHRIMAN_PORT=""
ENV AHRIMAN_POSTSETUP_COMMAND=""
ENV AHRIMAN_PRESETUP_COMMAND=""
ENV AHRIMAN_REPOSITORY="aur-clone"
ENV AHRIMAN_REPOSITORY_SERVER=""
ENV AHRIMAN_REPOSITORY_ROOT="/var/lib/ahriman/ahriman"
ENV AHRIMAN_UNIX_SOCKET=""
ENV AHRIMAN_USER="ahriman"
ENV AHRIMAN_VALIDATE_CONFIGURATION="yes"
# install environment
## update pacman.conf with multilib
RUN echo "[multilib]" >> "/etc/pacman.conf" && \
echo "Include = /etc/pacman.d/mirrorlist" >> "/etc/pacman.conf"
## refresh packages, install sudo and install packages for building
RUN pacman -Syu --noconfirm sudo && \
pacman -Sy --noconfirm --asdeps fakeroot python-tox
## create build user
RUN useradd -m -d "/home/build" -s "/usr/bin/nologin" build && \
echo "build ALL=(ALL) NOPASSWD: ALL" > "/etc/sudoers.d/build"
COPY "docker/install-aur-package.sh" "/usr/local/bin/install-aur-package"
## install package dependencies
## darcs is not installed by reasons, because it requires a lot haskell packages which dramatically increase image size
RUN pacman -Sy --noconfirm --asdeps devtools git pyalpm python-cerberus python-inflection python-passlib python-requests python-srcinfo && \
pacman -Sy --noconfirm --asdeps base-devel python-build python-flit python-installer python-wheel && \
pacman -Sy --noconfirm --asdeps breezy git mercurial python-aiohttp python-boto3 python-cryptography python-jinja python-requests-unixsocket python-systemd rsync subversion && \
runuser -u build -- install-aur-package python-aioauth-client python-webargs python-aiohttp-apispec-git python-aiohttp-cors \
python-aiohttp-jinja2 python-aiohttp-session python-aiohttp-security
## FIXME since 1.0.4 devtools requires dbus to be run, which doesn't work now in container
COPY "docker/systemd-nspawn.sh" "/usr/local/bin/systemd-nspawn"
# install ahriman
## copy tree
COPY --chown=build . "/home/build/ahriman"
## create package archive and install it
RUN cd "/home/build/ahriman" && \
tox -e archive && \
cp ./dist/*.tar.gz "package/archlinux" && \
cd "package/archlinux" && \
runuser -u build -- makepkg --noconfirm --install --skipchecksums && \
cd / && rm -r "/home/build/ahriman"
# cleanup unused
RUN find "/var/cache/pacman/pkg" -type f -delete
RUN pacman -Qdtq | pacman -Rscn --noconfirm -
VOLUME ["/var/lib/ahriman"]
# minimal runtime ahriman setup
COPY "docker/entrypoint.sh" "/usr/local/bin/entrypoint"
ENTRYPOINT ["entrypoint"]
# default command
CMD ["repo-update", "--refresh"]
+2 -4
View File
@@ -2,7 +2,7 @@
[![tests status](https://github.com/arcan1s/ahriman/actions/workflows/tests.yml/badge.svg)](https://github.com/arcan1s/ahriman/actions/workflows/tests.yml)
[![setup status](https://github.com/arcan1s/ahriman/actions/workflows/setup.yml/badge.svg)](https://github.com/arcan1s/ahriman/actions/workflows/setup.yml)
[![Docker Image Version (latest semver)](https://img.shields.io/docker/v/arcan1s/ahriman?label=Docker%20image&sort=semver)](https://hub.docker.com/r/arcan1s/ahriman)
[![Docker Image Version (latest semver)](https://img.shields.io/docker/v/arcan1s/ahriman?label=Docker%20image)](https://hub.docker.com/r/arcan1s/ahriman)
[![CodeFactor](https://www.codefactor.io/repository/github/arcan1s/ahriman/badge)](https://www.codefactor.io/repository/github/arcan1s/ahriman)
[![Documentation Status](https://readthedocs.org/projects/ahriman/badge/?version=latest)](https://ahriman.readthedocs.io)
@@ -33,12 +33,10 @@ Every available option is described in the [documentation](https://ahriman.readt
The application provides reasonable defaults which allow to use it out-of-box; however additional steps (like configuring build toolchain and sudoers) are recommended and can be easily achieved by following install instructions.
## [FAQ](https://ahriman.readthedocs.io/en/stable/faq/index.html)
## [FAQ](https://ahriman.readthedocs.io/en/stable/faq.html)
## Live demos
* [Build status page](https://ahriman-demo.arcanis.me). You can log in as `demo` user by using `demo` password. However, you will not be able to run tasks. [HTTP API documentation](https://ahriman-demo.arcanis.me/api-docs) is also available.
* [Repository index](https://repo.arcanis.me/arcanisrepo/x86_64/).
* [Telegram feed](https://t.me/arcanisrepo).
Do you have any success story? You can [share it](https://github.com/arcan1s/ahriman/issues/new?template=04-discussion.md)!
@@ -1,7 +0,0 @@
[archive]
; Keep amount of last built packages in archive. 0 means keep all packages
keep_built_packages = 1
[logs-rotation]
; Keep last build logs for each package
keep_last_logs = 5
@@ -1,27 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>{{ repository }}: Recent package updates</title>
{% if homepage is not none %}
<link>{{ homepage }}</link>
{% endif %}
<description>Recently updated packages in the {{ repository }}.</description>
{% if rss_url is not none %}
<atom:link href="{{ rss_url }}" rel="self"/>
{% endif %}
<language>en-us</language>
<lastBuildDate>{{ last_update }}</lastBuildDate>
{% for package in packages %}
<item>
<title>{{ package.name }} {{ package.version }} {{ package.architecture }}</title>
<link>{{ link_path }}/{{ package.filename }}</link>
<description>{{ package.description }}</description>
<pubDate>{{ package.build_date }}</pubDate>
<guid isPermaLink="false">{{ package.tag }}</guid>
<category>{{ repository }}</category>
<category>{{ package.architecture }}</category>
</item>
{% endfor %}
</channel>
</rss>
@@ -1,248 +0,0 @@
<script src="https://cdn.jsdelivr.net/npm/jquery@3.7.1/dist/jquery.min.js" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/js-md5@0.8.3/src/md5.min.js" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/tableexport.jquery.plugin@1.33.0/tableExport.min.js" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/jquery-resizable-columns@0.2.3/dist/jquery.resizableColumns.min.js" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/@popperjs/core@2.11.8/dist/umd/popper.min.js" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.7/dist/js/bootstrap.min.js" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap-table@1.24.1/dist/bootstrap-table.min.js" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap-table@1.24.1/dist/extensions/export/bootstrap-table-export.min.js" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap-table@1.24.1/dist/extensions/resizable/bootstrap-table-resizable.js" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap-table@1.24.1/dist/extensions/filter-control/bootstrap-table-filter-control.js" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap-table@1.24.1/dist/extensions/cookie/bootstrap-table-cookie.min.js" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/@easepick/bundle@1.2.1/dist/index.umd.min.js" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.11.1/build/highlight.min.js" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/chart.js@4.5.0/dist/chart.umd.min.js" crossorigin="anonymous" type="application/javascript"></script>
<script>
async function copyToClipboard(text, button) {
await navigator.clipboard.writeText(text);
button.innerHTML = "<i class=\"bi bi-clipboard-check\"></i> copied";
setTimeout(_ => {
button.innerHTML = "<i class=\"bi bi-clipboard\"></i> copy";
}, 2000);
}
function extractDataList(data, column) {
const elements = data.flatMap(row => row[column].split("<br>")).filter(v => v); // remove empty elements from array
return Array.from(new Set(elements)).sort();
}
function extractListProperties(description, property) {
return Object.values(description.packages)
.map(pkg => pkg[property])
.reduce((left, right) => left.concat(right), []);
}
function filterContains(text, value) {
return value.includes(text.toLowerCase().trim());
}
function filterDateRange(text, value) {
const asOfStartOfDay = date => date.setUTCHours(0, 0, 0, 0);
const [minDate, maxDate] = text.split(" - ");
const buildDate = asOfStartOfDay(new Date(value));
return (buildDate >= new Date(minDate)) && (buildDate <= new Date(maxDate));
}
function filterList(index, value, field, data) {
const dataList = extractDataList(data, field);
// the library removes all symbols from string, so it is just string
return value.includes(dataList[index].toLowerCase());
}
function hasActiveSelection() {
return !document.getSelection().isCollapsed; // not sure if it is a valid way, but I guess so
}
function hasActiveDropdown() {
return Array.from(document.querySelectorAll(".dropdown-menu"))
.some(el => el.classList.contains("show"));
}
function hasActiveModal() {
return Array.from(document.querySelectorAll(".modal"))
.some(el => el.classList.contains("show"));
}
function headerClass(status) {
if (status === "pending") return ["bg-warning"];
if (status === "building") return ["bg-warning"];
if (status === "failed") return ["bg-danger", "text-white"];
if (status === "success") return ["bg-success", "text-white"];
return ["bg-secondary", "text-white"];
}
function listToTable(data) {
return Array.from(new Set(data))
.sort()
.map(entry => safe(entry))
.join("<br>");
}
function makeRequest(url, params, onSuccess, onFailure) {
const requestParams = {
method: params.method,
body: params.json ? JSON.stringify(params.json) : params.json,
headers: {
"Accept": "application/json",
"Content-Type": "application/json",
},
};
if (params.query) {
const query = new URLSearchParams(params.query);
url += `?${query.toString()}`;
}
const convert = params.convert ?? (response => response.text());
return fetch(url, requestParams)
.then(response => {
if (response.ok) {
return convert(response);
} else {
const error = new Error("Network request error");
error.status = response.status;
error.statusText = response.statusText;
return response.text().then(text => {
error.text = text;
throw error;
});
}
})
.then(data => onSuccess && onSuccess(data))
.catch(error => onFailure && onFailure(error));
}
function readOptional(extractor, callback) {
for (let value = extractor(); !!value; value = null) {
callback(value);
}
}
function ready(fn) {
if (document.readyState === "complete" || document.readyState === "interactive") {
setTimeout(fn, 1);
} else {
document.addEventListener("DOMContentLoaded", fn);
}
}
function restoreAutoReloadSettings(toggle, intervalSelector) {
readOptional(() => localStorage.getItem(`ahriman-${toggle.id}-refresh-enabled`), value => toggle.checked = value === "true");
readOptional(() => localStorage.getItem(`ahriman-${toggle.id}-refresh-interval`), value => toggleActiveElement(intervalSelector, "interval", value));
}
function safe(string) {
return String(string)
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;");
}
// because I'm tired of safe element generation
function safeLink(url, text, title) {
const element = document.createElement("a");
element.href = url;
element.innerText = text;
if (title) {
element.title = title;
}
return element;
}
function toggleActiveElement(selector, dataType, value) {
const targetElement = selector.querySelector(`a[data-${dataType}="${value}"]`);
if (targetElement?.classList?.contains("active")) {
return; // element is already active, skip processing
}
Array.from(selector.children).forEach(il => {
Array.from(il.children).forEach(el => el.classList.remove("active"));
});
targetElement?.classList?.add("active");
}
function toggleAutoReload(toggle, interval, intervalSelector, callback) {
if (interval) {
toggle.checked = true; // toggle reload
} else {
interval = intervalSelector.querySelector(".active")?.dataset?.interval; // find active element
}
let intervalId = null;
if (interval) {
if (toggle.checked) {
// refresh UI
toggleActiveElement(intervalSelector, "interval", interval);
// finally create timer task
intervalId = setInterval(callback, interval);
}
} else {
toggle.checked = false; // no active interval found, disable toggle
}
localStorage.setItem(`ahriman-${toggle.id}-refresh-enabled`, toggle.checked);
localStorage.setItem(`ahriman-${toggle.id}-refresh-interval`, interval);
return intervalId;
}
function updateTable(table, rows, rowChangedKey) {
// instead of using load method here, we just update rows manually to avoid table reinitialization
const currentData = table.bootstrapTable("getData").reduce((accumulator, row) => {
accumulator[row.id] = {state: row["0"], key: rowChangedKey(row)};
return accumulator;
}, {});
// insert or update rows, skipping ones whose status hasn't changed
rows.forEach(row => {
if (Object.hasOwn(currentData, row.id)) {
if (rowChangedKey(row) === currentData[row.id].key) {
return;
}
row["0"] = currentData[row.id].state; // copy checkbox state
table.bootstrapTable("updateByUniqueId", {
id: row.id,
row: row,
replace: true,
});
} else {
table.bootstrapTable("insertRow", {index: 0, row: row});
}
});
// remove old rows
const newData = rows.map(value => value.id);
Object.keys(currentData).forEach(id => {
if (!newData.includes(id)) {
table.bootstrapTable("removeByUniqueId", id);
}
});
}
Array.prototype.equals = function (right, comparator) {
let index = this.length;
if (index !== right.length) {
return false;
}
while (index--) {
if (!comparator(this[index], right[index])) {
return false;
}
}
return true;
}
Date.prototype.toISOStringShort = function () {
const pad = number => String(number).padStart(2, "0");
return `${this.getFullYear()}-${pad(this.getMonth() + 1)}-${pad(this.getDate())} ${pad(this.getHours())}:${pad(this.getMinutes())}:${pad(this.getSeconds())}`;
}
</script>
-68
View File
@@ -1,68 +0,0 @@
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[project]
name = "ahriman-core"
description = "ArcH linux ReposItory MANager, core package"
readme = "../README.md"
requires-python = ">=3.13"
license = {file = "../COPYING"}
authors = [
{name = "ahriman team"},
]
dependencies = [
"bcrypt",
"filelock",
"inflection",
"pyelftools",
"requests",
]
dynamic = ["version"]
[project.optional-dependencies]
journald = [
"systemd-python",
]
# FIXME technically this dependency is required, but in some cases we do not have access to
# the libalpm which is required in order to install the package. Thus in case if we do not
# really need to run the application we can move it to "optional" dependencies
pacman = [
"pyalpm",
]
reports = [
"Jinja2",
]
s3 = [
"boto3",
]
shell = [
"IPython",
]
stats = [
"matplotlib",
]
unixsocket = [
"requests-unixsocket2",
]
validator = [
"cerberus",
]
[project.scripts]
ahriman = "ahriman.application.ahriman:run"
[project.urls]
Documentation = "https://ahriman.readthedocs.io/"
Repository = "https://github.com/arcan1s/ahriman"
Changelog = "https://github.com/arcan1s/ahriman/releases"
[tool.hatch.version]
path = "src/ahriman/__init__.py"
[tool.hatch.build.targets.wheel]
packages = ["src/ahriman"]
[tool.hatch.build.targets.wheel.shared-data]
"package/lib" = "lib"
"package/share" = "share"
@@ -1,120 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from pathlib import Path
import ahriman.application.handlers
from ahriman import __version__
from ahriman.application.handlers.handler import Handler
from ahriman.application.help_formatter import _HelpFormatter
from ahriman.core.module_loader import implementations
from ahriman.core.utils import enum_values
from ahriman.models.log_handler import LogHandler
__all__: list[str] = []
def _parser() -> argparse.ArgumentParser:
"""
command line parser generator
Returns:
argparse.ArgumentParser: command line parser for the application
"""
parser = argparse.ArgumentParser(prog="ahriman", description="ArcH linux ReposItory MANager",
epilog="""
Quick setup command (replace repository name, architecture and packager as needed):
>>> ahriman -a x86_64 -r aur service-setup --packager "ahriman bot <ahriman@example.com>"
Add new package from AUR:
>>> ahriman package-add ahriman --now
Check for updates and build out-of-dated packages (add ``--dry-run`` to build it later):
>>> ahriman repo-update
Remove package from the repository:
>>> ahriman package-remove ahriman
Start web service (requires additional configuration):
>>> ahriman web
""",
fromfile_prefix_chars="@", formatter_class=_HelpFormatter)
parser.add_argument("-a", "--architecture", help="filter by target architecture")
parser.add_argument("-c", "--configuration", help="configuration path", type=Path,
default=Path("/") / "etc" / "ahriman.ini")
parser.add_argument("--force", help="force run, remove file lock", action="store_true")
parser.add_argument("-l", "--lock", help="lock file", type=Path, default=Path("ahriman.pid"))
parser.add_argument("--log-handler", help="explicit log handler specification. If none set, the handler will be "
"guessed from environment",
type=LogHandler, choices=enum_values(LogHandler))
parser.add_argument("-q", "--quiet", help="force disable any logging", action="store_true")
parser.add_argument("--report", help="force enable or disable reporting to web service",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("-r", "--repository", help="filter by target repository")
# special secret argument for systemd unit. The issue is that systemd doesn't allow multiple arguments to template
# name. This parameter accepts [[arch]-repo] in order to keep backward compatibility
parser.add_argument("--repository-id", help=argparse.SUPPRESS)
parser.add_argument("--unsafe", help="allow to run ahriman as non-ahriman user. Some actions might be unavailable",
action="store_true")
parser.add_argument("-V", "--version", action="version", version=__version__)
parser.add_argument("--wait-timeout", help="wait for lock to be free. Negative value will lead to "
"immediate application run even if there is lock file. "
"In case of zero value, the application will wait infinitely",
type=int, default=-1)
subparsers = parser.add_subparsers(title="command", help="command to run", dest="command")
for handler in implementations(ahriman.application.handlers, Handler):
for subparser_parser in handler.arguments:
subparser = subparser_parser(subparsers)
subparser.formatter_class = _HelpFormatter
subparser.set_defaults(handler=handler, parser=_parser)
# sort actions alphabetically in both choices and help message
# pylint: disable=protected-access
subparsers._choices_actions = sorted(subparsers._choices_actions, key=lambda action: action.dest)
subparsers.choices = dict(sorted(subparsers.choices.items()))
return parser
def run() -> int:
"""
run application instance
Returns:
int: application status code
"""
parser = _parser()
args = parser.parse_args()
if args.command is None: # in case of empty command we would like to print help message
parser.exit(status=2, message=parser.format_help())
handler: Handler = args.handler
return handler.execute(args)
@@ -1,117 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from ahriman.application.application import Application
from ahriman.application.handlers.handler import Handler, SubParserAction
from ahriman.application.handlers.update import Update
from ahriman.core.configuration import Configuration
from ahriman.core.utils import enum_values, extract_user
from ahriman.models.package_source import PackageSource
from ahriman.models.pkgbuild_patch import PkgbuildPatch
from ahriman.models.repository_id import RepositoryId
class Add(Handler):
"""
add packages handler
"""
@classmethod
def run(cls, args: argparse.Namespace, repository_id: RepositoryId, configuration: Configuration, *,
report: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
"""
application = Application(repository_id, configuration, report=report, refresh_pacman_database=args.refresh)
application.on_start()
Add.perform_action(application, args)
@staticmethod
def _set_package_add_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for package addition subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("package-add", aliases=["add", "package-update"], help="add package",
description="add existing or new package to the build queue",
epilog="This subcommand should be used for new package addition. "
"It also supports flag --now in case if you would like to build "
"the package immediately. You can add new package from one of "
"supported sources:\n\n"
"1. If it is already built package you can specify the path to the archive.\n"
"2. You can also add built packages from the directory (e.g. during the "
"migration from another repository source).\n"
"3. It is also possible to add package from local PKGBUILD, but in this case "
"it will be ignored during the next automatic updates.\n"
"4. Ahriman supports downloading archives from remote (e.g. HTTP) sources.\n"
"5. Finally you can add package from AUR.")
parser.add_argument("package", help="package source (base name, path to local files, remote URL)", nargs="+")
parser.add_argument("--changes", help="calculate changes from the latest known commit if available",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--dependencies", help="process missing package dependencies",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty",
action="store_true")
parser.add_argument("--increment", help="increment package release (pkgrel) version on duplicate",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("-n", "--now", help="run update function after", action="store_true")
parser.add_argument("-s", "--source", help="explicitly specify the package source for this command",
type=PackageSource, choices=enum_values(PackageSource), default=PackageSource.Auto)
parser.add_argument("-u", "--username", help="build as user", default=extract_user())
parser.add_argument("-v", "--variable", help="apply specified makepkg variables to the next build",
action="append")
parser.add_argument("-y", "--refresh", help="download fresh package databases from the mirror before actions, "
"-yy to force refresh even if up to date",
action="count", default=False)
parser.set_defaults(aur=False, check_files=False, dry_run=False, local=False, manual=True, vcs=False)
return parser
@staticmethod
def perform_action(application: Application, args: argparse.Namespace) -> None:
"""
perform add action
Args:
application(Application): application instance
args(argparse.Namespace): command line args
"""
application.add(args.package, args.source, args.username)
patches = [PkgbuildPatch.from_env(patch) for patch in args.variable] if args.variable is not None else []
for package in args.package: # for each requested package insert patch
for patch in patches:
application.reporter.package_patches_update(package, patch)
if not args.now:
return
Update.perform_action(application, args)
arguments = [_set_package_add_parser]
@@ -1,81 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from ahriman.application.application import Application
from ahriman.application.handlers.handler import Handler, SubParserAction
from ahriman.core.configuration import Configuration
from ahriman.core.formatters import PackagePrinter
from ahriman.models.action import Action
from ahriman.models.build_status import BuildStatus, BuildStatusEnum
from ahriman.models.repository_id import RepositoryId
class Archives(Handler):
"""
package archives handler
"""
ALLOW_MULTI_ARCHITECTURE_RUN = False # conflicting io
@classmethod
def run(cls, args: argparse.Namespace, repository_id: RepositoryId, configuration: Configuration, *,
report: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
"""
application = Application(repository_id, configuration, report=True)
match args.action:
case Action.List:
archives = application.repository.package_archives(args.package)
for package in archives:
PackagePrinter(package, BuildStatus(BuildStatusEnum.Success))(verbose=args.info)
Archives.check_status(args.exit_code, bool(archives))
@staticmethod
def _set_package_archives_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for package archives subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("package-archives", help="list package archive versions",
description="list available archive versions for the package")
parser.add_argument("package", help="package base")
parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty",
action="store_true")
parser.add_argument("--info", help="show additional package information",
action=argparse.BooleanOptionalAction, default=False)
parser.set_defaults(action=Action.List, lock=None, quiet=True, report=False, unsafe=True)
return parser
arguments = [_set_package_archives_parser]
@@ -1,98 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from ahriman.application.application import Application
from ahriman.application.handlers.handler import Handler, SubParserAction
from ahriman.core.configuration import Configuration
from ahriman.core.formatters import ChangesPrinter
from ahriman.models.action import Action
from ahriman.models.changes import Changes
from ahriman.models.repository_id import RepositoryId
class Change(Handler):
"""
package changes handler
"""
ALLOW_MULTI_ARCHITECTURE_RUN = False # conflicting io
@classmethod
def run(cls, args: argparse.Namespace, repository_id: RepositoryId, configuration: Configuration, *,
report: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
"""
client = Application(repository_id, configuration, report=True).reporter
match args.action:
case Action.List:
changes = client.package_changes_get(args.package)
ChangesPrinter(changes)(verbose=True, separator="")
Change.check_status(args.exit_code, changes.changes is not None)
case Action.Remove:
client.package_changes_update(args.package, Changes())
@staticmethod
def _set_package_changes_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for package changes subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("package-changes", help="get package changes",
description="retrieve package changes stored in database",
epilog="This command requests package status from the web interface "
"if it is available.")
parser.add_argument("package", help="package base")
parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty",
action="store_true")
parser.set_defaults(action=Action.List, lock=None, quiet=True, report=False, unsafe=True)
return parser
@staticmethod
def _set_package_changes_remove_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for package change remove subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("package-changes-remove", help="remove package changes",
description="remove the package changes stored remotely")
parser.add_argument("package", help="package base")
parser.set_defaults(action=Action.Remove, exit_code=False, lock=None, quiet=True, report=False, unsafe=True)
return parser
arguments = [_set_package_changes_parser, _set_package_changes_remove_parser]
@@ -1,78 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from ahriman.application.application import Application
from ahriman.application.handlers.handler import Handler, SubParserAction
from ahriman.core.configuration import Configuration
from ahriman.models.repository_id import RepositoryId
class Clean(Handler):
"""
clean caches handler
"""
@classmethod
def run(cls, args: argparse.Namespace, repository_id: RepositoryId, configuration: Configuration, *,
report: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
"""
application = Application(repository_id, configuration, report=report)
application.on_start()
application.clean(cache=args.cache, chroot=args.chroot, manual=args.manual, packages=args.packages,
pacman=args.pacman)
@staticmethod
def _set_service_clean_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for repository clean subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("service-clean", aliases=["clean", "repo-clean"], help="clean local caches",
description="remove local caches",
epilog="The subcommand clears every temporary directories (builds, caches etc). "
"Normally you should not run this command manually. Also in case if "
"you are going to clear the chroot directories you will need root privileges.")
parser.add_argument("--cache", help="clear directory with package caches",
action=argparse.BooleanOptionalAction, default=False)
parser.add_argument("--chroot", help="clear build chroot", action=argparse.BooleanOptionalAction, default=False)
parser.add_argument("--manual", help="clear manually added packages queue",
action=argparse.BooleanOptionalAction, default=False)
parser.add_argument("--packages", help="clear directory with built packages",
action=argparse.BooleanOptionalAction, default=False)
parser.add_argument("--pacman", help="clear directory with pacman local database cache",
action=argparse.BooleanOptionalAction, default=False)
parser.set_defaults(lock=None, quiet=True, unsafe=True)
return parser
arguments = [_set_service_clean_parser]
@@ -1,117 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from ahriman.application.application import Application
from ahriman.application.handlers.handler import Handler, SubParserAction
from ahriman.core.configuration import Configuration
from ahriman.models.build_status import BuildStatusEnum
from ahriman.models.package import Package
from ahriman.models.package_source import PackageSource
from ahriman.models.repository_id import RepositoryId
class Copy(Handler):
"""
copy packages handler
"""
ALLOW_MULTI_ARCHITECTURE_RUN = False # conflicting action
@classmethod
def run(cls, args: argparse.Namespace, repository_id: RepositoryId, configuration: Configuration, *,
report: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
"""
application = Application(repository_id, configuration, report=report)
application.on_start()
configuration_path, _ = configuration.check_loaded()
source_repository_id = RepositoryId(repository_id.architecture, args.source)
source_configuration = Configuration.from_path(configuration_path, source_repository_id)
source_application = Application(source_repository_id, source_configuration, report=report)
packages = source_application.repository.packages(args.package)
Copy.check_status(args.exit_code, packages)
for package in packages:
Copy.copy_package(package, application, source_application)
# run update
application.update([])
if args.remove:
source_application.remove(args.package)
@staticmethod
def _set_package_copy_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for package copy subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("package-copy", aliases=["copy"], help="copy package from another repository",
description="copy package and its metadata from another repository")
parser.add_argument("source", help="source repository name")
parser.add_argument("package", help="package base", nargs="+")
parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty",
action="store_true")
parser.add_argument("--remove", help="remove package from the source repository after", action="store_true")
return parser
@staticmethod
def copy_package(package: Package, application: Application, source_application: Application) -> None:
"""
copy package ``package`` from source repository to target repository
Args:
package(Package): package to copy
application(Application): application instance of the target repository
source_application(Application): application instance of the source repository
"""
# copy files
source_paths = [
str(source_application.repository.paths.repository / source.filename)
for source in package.packages.values()
if source.filename is not None
]
application.add(source_paths, PackageSource.Archive)
# copy metadata
application.reporter.package_changes_update(
package.base, source_application.reporter.package_changes_get(package.base)
)
application.reporter.package_dependencies_update(
package.base, source_application.reporter.package_dependencies_get(package.base)
)
application.reporter.package_update(package, BuildStatusEnum.Pending)
arguments = [_set_package_copy_parser]
@@ -1,106 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from pathlib import Path
from ahriman.application.application import Application
from ahriman.application.application.updates_iterator import FixedUpdatesIterator, UpdatesIterator
from ahriman.application.handlers.handler import Handler, SubParserAction
from ahriman.application.handlers.update import Update
from ahriman.core.configuration import Configuration
from ahriman.core.utils import extract_user
from ahriman.models.repository_id import RepositoryId
class Daemon(Handler):
"""
daemon packages handler
"""
@classmethod
def run(cls, args: argparse.Namespace, repository_id: RepositoryId, configuration: Configuration, *,
report: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
"""
application = Application(repository_id, configuration, report=report, refresh_pacman_database=args.refresh)
if args.partitions:
iterator = UpdatesIterator(application, args.interval)
else:
iterator = FixedUpdatesIterator(application, args.interval)
for packages in iterator:
if packages is None:
continue # nothing to check case
args.package = packages
Update.run(args, repository_id, configuration, report=report)
@staticmethod
def _set_repo_daemon_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for daemon subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("repo-daemon", aliases=["daemon"], help="run application as daemon",
description="start process which periodically will run update process")
parser.add_argument("-i", "--interval", help="interval between runs in seconds", type=int, default=60 * 60 * 12)
parser.add_argument("--aur", help="enable or disable checking for AUR updates",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--changes", help="calculate changes from the latest known commit if available. "
"Only applicable in dry run mode",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--check-files", help="enable or disable checking of broken dependencies "
"(e.g. dynamically linked libraries or modules directories)",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--dependencies", help="process missing package dependencies",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--dry-run", help="just perform check for updates, same as check command",
action="store_true")
parser.add_argument("--increment", help="increment package release (pkgrel) on duplicate",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--local", help="enable or disable checking of local packages for updates",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--manual", help="include or exclude manual updates",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--partitions", help="instead of updating whole repository, split updates into chunks",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("-u", "--username", help="build as user", default=extract_user())
parser.add_argument("--vcs", help="fetch actual version of VCS packages",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("-y", "--refresh", help="download fresh package databases from the mirror before actions, "
"-yy to force refresh even if up to date",
action="count", default=False)
parser.set_defaults(exit_code=False, lock=Path("ahriman-daemon.pid"), package=[])
return parser
arguments = [_set_repo_daemon_parser]
@@ -1,93 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from ahriman.application.application import Application
from ahriman.application.handlers.handler import Handler, SubParserAction
from ahriman.core.configuration import Configuration
from ahriman.models.action import Action
from ahriman.models.repository_id import RepositoryId
class Hold(Handler):
"""
package hold handler
"""
@classmethod
def run(cls, args: argparse.Namespace, repository_id: RepositoryId, configuration: Configuration, *,
report: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
"""
client = Application(repository_id, configuration, report=True).reporter
match args.action:
case Action.Remove:
for package in args.package:
client.package_hold_update(package, enabled=False)
case Action.Update:
for package in args.package:
client.package_hold_update(package, enabled=True)
@staticmethod
def _set_package_hold_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for hold package subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("package-hold", help="hold package",
description="hold package from automatic updates")
parser.add_argument("package", help="package base", nargs="+")
parser.set_defaults(action=Action.Update, lock=None, quiet=True, report=False, unsafe=True)
return parser
@staticmethod
def _set_package_unhold_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for unhold package subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("package-unhold", help="unhold package",
description="remove package hold, allowing automatic updates")
parser.add_argument("package", help="package base", nargs="+")
parser.set_defaults(action=Action.Remove, lock=None, quiet=True, report=False, unsafe=True)
return parser
arguments = [
_set_package_hold_parser,
_set_package_unhold_parser,
]
@@ -1,72 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from ahriman.application.application import Application
from ahriman.application.handlers.handler import Handler, SubParserAction
from ahriman.core.configuration import Configuration
from ahriman.models.repository_id import RepositoryId
class KeyImport(Handler):
"""
key import packages handler
"""
ALLOW_MULTI_ARCHITECTURE_RUN = False # system-wide action
@classmethod
def run(cls, args: argparse.Namespace, repository_id: RepositoryId, configuration: Configuration, *,
report: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
"""
application = Application(repository_id, configuration, report=report)
application.repository.sign.key_import(args.key_server, args.key)
@staticmethod
def _set_service_key_import_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for key import subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("service-key-import", aliases=["key-import"], help="import PGP key",
description="import PGP key from public sources to the repository user",
epilog="By default ahriman runs build process with package sources validation "
"(in case if signature and keys are available in PKGBUILD). This process will "
"fail in case if key is not known for build user. This subcommand can be used "
"in order to import the PGP key to user keychain.")
parser.add_argument("--key-server", help="key server for key import", default="keyserver.ubuntu.com")
parser.add_argument("key", help="PGP key to import from public server")
parser.set_defaults(architecture="", lock=None, report=False, repository="")
return parser
arguments = [_set_service_key_import_parser]
@@ -1,257 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
import sys
from pathlib import Path
from ahriman.application.application import Application
from ahriman.application.handlers.handler import Handler, SubParserAction
from ahriman.core.build_tools.sources import Sources
from ahriman.core.configuration import Configuration
from ahriman.core.formatters import PatchPrinter
from ahriman.models.action import Action
from ahriman.models.package import Package
from ahriman.models.pkgbuild_patch import PkgbuildPatch
from ahriman.models.repository_id import RepositoryId
class Patch(Handler):
"""
patch control handler
"""
ALLOW_MULTI_ARCHITECTURE_RUN = False # system-wide action
@classmethod
def run(cls, args: argparse.Namespace, repository_id: RepositoryId, configuration: Configuration, *,
report: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
"""
application = Application(repository_id, configuration, report=report)
application.on_start()
match args.action:
case Action.Update if args.variable is not None:
patch = Patch.patch_create_from_function(args.variable, args.patch)
Patch.patch_set_create(application, args.package, patch)
case Action.Update:
package_base, patch = Patch.patch_create_from_diff(args.package, repository_id.architecture, args.track)
Patch.patch_set_create(application, package_base, patch)
case Action.List:
Patch.patch_set_list(application, args.package, args.variable, args.exit_code)
case Action.Remove:
Patch.patch_set_remove(application, args.package, args.variable)
@staticmethod
def _set_patch_add_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for new single-function patch subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("patch-add", help="add patch for PKGBUILD function",
description="create or update patched PKGBUILD function or variable",
epilog="Unlike ``patch-set-add``, this function allows to patch only one PKGBUILD "
"function, e.g. typing ``ahriman patch-add ahriman pkgver`` it will change the "
"``pkgver`` inside PKGBUILD, typing ``ahriman patch-add ahriman build()`` "
"it will change ``build()`` function inside PKGBUILD.")
parser.add_argument("package", help="package base")
parser.add_argument("variable", help="PKGBUILD variable or function name. If variable is a function, "
"it must end with ()")
parser.add_argument("patch", help="path to file which contains function or variable value. If not set, "
"the value will be read from stdin", type=Path, nargs="?")
parser.set_defaults(action=Action.Update, architecture="", exit_code=False, lock=None, report=False,
repository="")
return parser
@staticmethod
def _set_patch_list_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for list patches subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("patch-list", help="list patch sets",
description="list available patches for the package")
parser.add_argument("package", help="package base")
parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty",
action="store_true")
parser.add_argument("-v", "--variable", help="if set, show only patches for specified PKGBUILD variables",
action="append")
parser.set_defaults(action=Action.List, architecture="", lock=None, report=False, repository="", unsafe=True)
return parser
@staticmethod
def _set_patch_remove_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for remove patches subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("patch-remove", help="remove patch set", description="remove patches for the package")
parser.add_argument("package", help="package base")
parser.add_argument("-v", "--variable",
help="should be used for single-function patches in case if you wold like "
"to remove only specified PKGBUILD variables. In case if not set, "
"it will remove all patches related to the package",
action="append")
parser.set_defaults(action=Action.Remove, architecture="", exit_code=False, lock=None, report=False,
repository="")
return parser
@staticmethod
def _set_patch_set_add_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for new full-diff patch subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("patch-set-add", help="add patch set", description="create or update source patches",
epilog="In order to add a patch set for the package you will need to:\n\n"
"1. Clone the AUR package manually.\n"
"2. Add required changes (e.g. external patches, edit PKGBUILD).\n"
"3. Run command, e.g. ``ahriman patch-set-add path/to/directory``.\n\n"
"By default it tracks ``*.patch`` and ``*.diff`` files, but this behavior "
"can be changed by using ``--track`` option.")
parser.add_argument("package", help="path to directory with changed files for patch addition/update", type=Path)
parser.add_argument("-t", "--track", help="files which has to be tracked", action="append",
default=["*.diff", "*.patch"])
parser.set_defaults(action=Action.Update, architecture="", exit_code=False, lock=None, report=False,
repository="", variable=None)
return parser
@staticmethod
def patch_create_from_diff(sources_dir: Path, architecture: str, track: list[str]) -> tuple[str, PkgbuildPatch]:
"""
create PKGBUILD plain diff patches from sources directory
Args:
sources_dir(Path): path to directory with the package sources
architecture(str): repository architecture
track(list[str]): track files which match the glob before creating the patch
Returns:
tuple[str, PkgbuildPatch]: package base and created PKGBUILD patch based on the diff from master HEAD
to current files
"""
package = Package.from_build(sources_dir, architecture, None)
patch = Sources.patch_create(sources_dir, *track)
return package.base, PkgbuildPatch(None, patch)
@staticmethod
def patch_create_from_function(variable: str, patch_path: Path | None) -> PkgbuildPatch:
"""
create single-function patch set for the package base
Args:
variable(str): function or variable name inside PKGBUILD
patch_path(Path | None): optional path to patch content. If not set, it will be read from stdin
Returns:
PkgbuildPatch: created patch for the PKGBUILD function
"""
if patch_path is None:
# pylint: disable=bad-builtin
print("Post new function or variable value below. Press Ctrl-D to finish:", file=sys.stderr)
patch = "".join(list(sys.stdin))
else:
patch = patch_path.read_text(encoding="utf8")
# remove spaces around the patch and parse to correct type
return PkgbuildPatch.parse(variable, patch.strip())
@staticmethod
def patch_set_create(application: Application, package_base: str, patch: PkgbuildPatch) -> None:
"""
create patch set for the package base
Args:
application(Application): application instance
package_base(str): package base
patch(PkgbuildPatch): patch descriptor
"""
application.reporter.package_patches_update(package_base, patch)
@staticmethod
def patch_set_list(application: Application, package_base: str, variables: list[str] | None,
exit_code: bool) -> None:
"""
list patches available for the package base
Args:
application(Application): application instance
package_base(str): package base
variables(list[str] | None): extract patches only for specified PKGBUILD variables
exit_code(bool): exit with error on empty search result
"""
patches = [
patch
for patch in application.reporter.package_patches_get(package_base, None)
if variables is None or patch.key in variables
]
Patch.check_status(exit_code, patches)
PatchPrinter(package_base, patches)(verbose=True, separator=" = ")
@staticmethod
def patch_set_remove(application: Application, package_base: str, variables: list[str] | None) -> None:
"""
remove patch set for the package base
Args:
application(Application): application instance
package_base(str): package base
variables(list[str] | None): remove patches only for specified PKGBUILD variables
"""
if variables is not None:
for variable in variables: # iterate over single variable
application.reporter.package_patches_remove(package_base, variable)
else:
application.reporter.package_patches_remove(package_base, None) # just pass as is
arguments = [
_set_patch_add_parser,
_set_patch_list_parser,
_set_patch_remove_parser,
_set_patch_set_add_parser,
]
@@ -1,100 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from dataclasses import replace
from ahriman.application.application import Application
from ahriman.application.handlers.handler import Handler, SubParserAction
from ahriman.core.configuration import Configuration
from ahriman.core.formatters import PkgbuildPrinter
from ahriman.models.action import Action
from ahriman.models.repository_id import RepositoryId
class Pkgbuild(Handler):
"""
package pkgbuild handler
"""
ALLOW_MULTI_ARCHITECTURE_RUN = False # conflicting io
@classmethod
def run(cls, args: argparse.Namespace, repository_id: RepositoryId, configuration: Configuration, *,
report: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
"""
client = Application(repository_id, configuration, report=True).reporter
match args.action:
case Action.List:
changes = client.package_changes_get(args.package)
PkgbuildPrinter(changes)(verbose=True, separator="")
Pkgbuild.check_status(args.exit_code, changes.pkgbuild is not None)
case Action.Remove:
changes = client.package_changes_get(args.package)
client.package_changes_update(args.package, replace(changes, pkgbuild=None))
@staticmethod
def _set_package_pkgbuild_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for package pkgbuild subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("package-pkgbuild", help="get package pkgbuild",
description="retrieve package PKGBUILD stored in database",
epilog="This command requests package status from the web interface "
"if it is available.")
parser.add_argument("package", help="package base")
parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty",
action="store_true")
parser.set_defaults(action=Action.List, lock=None, quiet=True, report=False, unsafe=True)
return parser
@staticmethod
def _set_package_pkgbuild_remove_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for package pkgbuild remove subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("package-pkgbuild-remove", help="remove package pkgbuild",
description="remove the package PKGBUILD stored remotely")
parser.add_argument("package", help="package base")
parser.set_defaults(action=Action.Remove, exit_code=False, lock=None, quiet=True, report=False, unsafe=True)
return parser
arguments = [_set_package_pkgbuild_parser, _set_package_pkgbuild_remove_parser]
@@ -1,118 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from ahriman.application.application import Application
from ahriman.application.handlers.handler import Handler, SubParserAction
from ahriman.core.configuration import Configuration
from ahriman.core.utils import enum_values, extract_user
from ahriman.models.build_status import BuildStatusEnum
from ahriman.models.package import Package
from ahriman.models.packagers import Packagers
from ahriman.models.repository_id import RepositoryId
class Rebuild(Handler):
"""
make world handler
"""
@classmethod
def run(cls, args: argparse.Namespace, repository_id: RepositoryId, configuration: Configuration, *,
report: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
"""
application = Application(repository_id, configuration, report=report)
application.on_start()
packages = Rebuild.extract_packages(application, args.status, from_database=args.from_database)
packages = application.repository.packages_depend_on(packages, args.depends_on)
Rebuild.check_status(args.exit_code, packages)
if args.dry_run:
application.print_updates(packages, log_fn=print)
return
result = application.update(packages, Packagers(args.username), bump_pkgrel=args.increment)
Rebuild.check_status(args.exit_code, not result.is_empty)
@staticmethod
def _set_repo_rebuild_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for repository rebuild subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("repo-rebuild", aliases=["rebuild"], help="rebuild repository",
description="force rebuild whole repository")
parser.add_argument("--depends-on", help="only rebuild packages that depend on specified packages",
action="append")
parser.add_argument("--dry-run", help="just perform check for packages without rebuild process itself",
action="store_true")
parser.add_argument("--from-database",
help="read packages from database instead of filesystem. This feature in particular is "
"required in case if you would like to restore repository from another repository "
"instance. Note, however, that in order to restore packages you need to have original "
"ahriman instance run with web service and have run repo-update at least once.",
action="store_true")
parser.add_argument("--increment", help="increment package release (pkgrel) on duplicate",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty",
action="store_true")
parser.add_argument("-s", "--status", help="filter packages by status. Requires --from-database to be set",
type=BuildStatusEnum, choices=enum_values(BuildStatusEnum))
parser.add_argument("-u", "--username", help="build as user", default=extract_user())
return parser
@staticmethod
def extract_packages(application: Application, status: BuildStatusEnum | None, *,
from_database: bool) -> list[Package]:
"""
extract packages from database file
Args:
application(Application): application instance
status(BuildStatusEnum | None): optional filter by package status
from_database(bool): extract packages from database instead of repository filesystem
Returns:
list[Package]: list of packages which were stored in database
"""
if from_database:
return [
package
for (package, last_status) in application.reporter.package_get(None)
if status is None or last_status.status == status
]
return application.repository.packages()
arguments = [_set_repo_rebuild_parser]
@@ -1,131 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from dataclasses import replace
from pathlib import Path
from ahriman.application.application import Application
from ahriman.application.handlers.add import Add
from ahriman.application.handlers.handler import Handler, SubParserAction
from ahriman.core.configuration import Configuration
from ahriman.core.exceptions import UnknownPackageError
from ahriman.core.utils import extract_user
from ahriman.models.package import Package
from ahriman.models.package_source import PackageSource
from ahriman.models.repository_id import RepositoryId
class Rollback(Handler):
"""
package rollback handler
"""
@classmethod
def run(cls, args: argparse.Namespace, repository_id: RepositoryId, configuration: Configuration, *,
report: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
"""
application = Application(repository_id, configuration, report=report)
application.on_start()
package = Rollback.package_load(application, args.package, args.version)
artifacts = Rollback.package_artifacts(application, package)
args.package = [str(artifact) for artifact in artifacts]
Add.perform_action(application, args)
if args.hold:
application.reporter.package_hold_update(package.base, enabled=True)
@staticmethod
def _set_package_rollback_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for package rollback subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("package-rollback", help="rollback package",
description="rollback package to specified version from archives")
parser.add_argument("package", help="package base")
parser.add_argument("version", help="package version")
parser.add_argument("--hold", help="hold package afterwards",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("-u", "--username", help="build as user", default=extract_user())
parser.set_defaults(aur=False, changes=False, check_files=False, dependencies=False, dry_run=False,
exit_code=True, increment=False, now=True, local=False, manual=False, refresh=False,
source=PackageSource.Archive, variable=None, vcs=False)
return parser
@staticmethod
def package_artifacts(application: Application, package: Package) -> list[Path]:
"""
look for requested package artifacts and return paths to them
Args:
application(Application): application instance
package(Package): package descriptor
Returns:
list[Path]: paths to found artifacts
Raises:
UnknownPackageError: if artifacts do not exist
"""
# lookup for built artifacts
artifacts = application.repository.package_archives_lookup(package)
if not artifacts:
raise UnknownPackageError(package.base)
return artifacts
@staticmethod
def package_load(application: Application, package_base: str, version: str) -> Package:
"""
load package from repository, while setting requested version
Args:
application(Application): application instance
package_base(str): package base
version(str): package version
Returns:
Package: loaded package
Raises:
UnknownPackageError: if package does not exist
"""
try:
package, _ = next(iter(application.reporter.package_get(package_base)))
return replace(package, version=version)
except StopIteration:
raise UnknownPackageError(package_base) from None
arguments = [_set_package_rollback_parser]
@@ -1,197 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
import datetime
import itertools
from collections.abc import Callable
from pathlib import Path
from ahriman.application.application import Application
from ahriman.application.handlers.handler import Handler, SubParserAction
from ahriman.core.configuration import Configuration
from ahriman.core.formatters import EventStatsPrinter, PackageStatsPrinter, RepositoryStatsPrinter
from ahriman.core.utils import enum_values, pretty_datetime
from ahriman.models.event import Event, EventType
from ahriman.models.repository_id import RepositoryId
class Statistics(Handler):
"""
repository statistics handler
"""
ALLOW_MULTI_ARCHITECTURE_RUN = False # conflicting io
@classmethod
def run(cls, args: argparse.Namespace, repository_id: RepositoryId, configuration: Configuration, *,
report: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
"""
application = Application(repository_id, configuration, report=True)
from_date = to_date = None
if (value := args.from_date) is not None:
from_date = datetime.datetime.fromisoformat(value).timestamp()
if (value := args.to_date) is not None:
to_date = datetime.datetime.fromisoformat(value).timestamp()
events = application.reporter.event_get(args.event, args.package, from_date, to_date, args.limit, args.offset)
match args.package:
case None:
RepositoryStatsPrinter(repository_id, application.reporter.statistics())(verbose=True)
Statistics.stats_per_package(args.event, events, args.chart)
case _:
Statistics.stats_for_package(args.event, events, args.chart)
@staticmethod
def _set_repo_statistics_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for repository statistics subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("repo-statistics", help="repository statistics",
description="fetch repository statistics")
parser.add_argument("package", help="fetch only events for the specified package", nargs="?")
parser.add_argument("--chart", help="create updates chart and save it to the specified path", type=Path)
parser.add_argument("-e", "--event", help="event type filter",
type=EventType, choices=enum_values(EventType), default=EventType.PackageUpdated)
parser.add_argument("--from-date", help="only fetch events which are newer than the date")
parser.add_argument("--limit", help="limit response by specified amount of events", type=int, default=-1)
parser.add_argument("--offset", help="skip specified amount of events", type=int, default=0)
parser.add_argument("--to-date", help="only fetch events which are older than the date")
parser.set_defaults(lock=None, quiet=True, report=False, unsafe=True)
return parser
@staticmethod
def event_stats(event_type: str, events: list[Event]) -> None:
"""
calculate event stats
Args:
event_type(str): event type
events(list[Event]): list of events
"""
times = [event.get("took") for event in events if event.get("took") is not None]
EventStatsPrinter(f"{event_type} duration, s", times)(verbose=True)
@staticmethod
def plot_packages(event_type: str, events: dict[str, int], path: Path) -> None:
"""
plot packages frequency
Args:
event_type(str): event type
events(dict[str, int]): list of events
path(Path): path to save plot
"""
from matplotlib import pyplot as plt
x, y = list(events.keys()), list(events.values())
plt.bar(x, y)
plt.xlabel("Package base")
plt.ylabel("Frequency")
plt.title(f"Frequency of the {event_type} event per package")
plt.savefig(path)
@staticmethod
def plot_times(event_type: str, events: list[Event], path: Path) -> None:
"""
plot events timeline
Args:
event_type(str): event type
events(list[Event]): list of events
path(Path): path to save plot
"""
from matplotlib import pyplot as plt
figure = plt.figure()
x, y = zip(*[(pretty_datetime(event.created), event.get("took")) for event in events])
plt.plot(x, y)
plt.xlabel("Event timestamp")
plt.ylabel("Duration, s")
plt.title(f"Duration of the {event_type} event")
figure.autofmt_xdate()
plt.savefig(path)
@staticmethod
def stats_for_package(event_type: str, events: list[Event], chart_path: Path | None) -> None:
"""
calculate statistics for a package
Args:
event_type(str): event type
events(list[Event]): list of events
chart_path(Path): path to save plot if any
"""
# event statistics
Statistics.event_stats(event_type, events)
# chart if enabled
if chart_path is not None:
Statistics.plot_times(event_type, events, chart_path)
@staticmethod
def stats_per_package(event_type: str, events: list[Event], chart_path: Path | None) -> None:
"""
calculate overall statistics
Args:
event_type(str): event type
events(list[Event]): list of events
chart_path(Path): path to save plot if any
"""
key: Callable[[Event], str] = lambda event: event.object_id
by_object_id = {
object_id: len(list(related))
for object_id, related in itertools.groupby(sorted(events, key=key), key=key)
}
# distribution per package
PackageStatsPrinter(by_object_id)(verbose=True)
EventStatsPrinter(f"{event_type} frequency", list(by_object_id.values()))(verbose=True)
# event statistics
Statistics.event_stats(event_type, events)
# chart if enabled
if chart_path is not None:
Statistics.plot_packages(event_type, by_object_id, chart_path)
arguments = [_set_repo_statistics_parser]
@@ -1,126 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from ahriman.application.application import Application
from ahriman.application.handlers.handler import Handler, SubParserAction
from ahriman.core.configuration import Configuration
from ahriman.core.utils import enum_values
from ahriman.models.action import Action
from ahriman.models.build_status import BuildStatusEnum
from ahriman.models.repository_id import RepositoryId
class StatusUpdate(Handler):
"""
status update handler
"""
ALLOW_MULTI_ARCHITECTURE_RUN = False # conflicting io
@classmethod
def run(cls, args: argparse.Namespace, repository_id: RepositoryId, configuration: Configuration, *,
report: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
"""
client = Application(repository_id, configuration, report=True).reporter
match args.action:
case Action.Update if args.package:
# update packages statuses
for package in args.package:
client.package_status_update(package, args.status)
case Action.Update:
# update service status
client.status_update(args.status)
case Action.Remove:
for package in args.package:
client.package_remove(package)
@staticmethod
def _set_package_status_remove_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for package status remove subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("package-status-remove", help="remove package status",
description="remove the package from the status page",
epilog="Please note that this subcommand does not remove the package itself, it just "
"clears the status page.")
parser.add_argument("package", help="remove specified packages from status page", nargs="+")
parser.set_defaults(action=Action.Remove, lock=None, quiet=True, report=False, unsafe=True)
return parser
@staticmethod
def _set_package_status_update_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for package status update subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("package-status-update", aliases=["status-update"], help="update package status",
description="update package status on the status page")
parser.add_argument("package", help="set status for specified packages. "
"If no packages supplied, service status will be updated",
nargs="*")
parser.add_argument("-s", "--status", help="new package build status",
type=BuildStatusEnum, choices=enum_values(BuildStatusEnum), default=BuildStatusEnum.Success)
parser.set_defaults(action=Action.Update, lock=None, quiet=True, report=False, unsafe=True)
return parser
@staticmethod
def _set_repo_status_update_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for repository status update subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("repo-status-update", help="update repository status",
description="update repository status on the status page")
parser.add_argument("-s", "--status", help="new status",
type=BuildStatusEnum, choices=enum_values(BuildStatusEnum), default=BuildStatusEnum.Success)
parser.set_defaults(action=Action.Update, lock=None, package=[], quiet=True, report=False, unsafe=True)
return parser
arguments = [
_set_package_status_remove_parser,
_set_package_status_update_parser,
_set_repo_status_update_parser,
]
@@ -1,108 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from ahriman.application.application import Application
from ahriman.application.handlers.handler import Handler, SubParserAction
from ahriman.core.configuration import Configuration
from ahriman.models.repository_id import RepositoryId
from ahriman.models.result import Result
class Triggers(Handler):
"""
triggers handlers
"""
@classmethod
def run(cls, args: argparse.Namespace, repository_id: RepositoryId, configuration: Configuration, *,
report: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
"""
application = Application(repository_id, configuration, report=report)
if args.trigger:
loader = application.repository.triggers
loader.triggers = [loader.load_trigger(trigger, repository_id, configuration) for trigger in args.trigger]
application.on_start()
application.on_result(Result())
@staticmethod
def _set_repo_report_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for report subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("repo-report", aliases=["report"], help="generate report",
description="generate repository report according to current settings",
epilog="Create and/or update repository report as configured.")
parser.set_defaults(trigger=["ahriman.core.report.ReportTrigger"])
return parser
@staticmethod
def _set_repo_sync_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for repository sync subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("repo-sync", aliases=["sync"], help="sync repository",
description="sync repository files to remote server according to current settings",
epilog="Synchronize the repository to remote services as configured.")
parser.set_defaults(trigger=["ahriman.core.upload.UploadTrigger"])
return parser
@staticmethod
def _set_repo_triggers_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for repository triggers subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("repo-triggers", help="run triggers",
description="run triggers on empty build result as configured by settings")
parser.add_argument("trigger", help="instead of running all triggers as set by configuration, just process "
"specified ones in order of mention", nargs="*")
return parser
arguments = [
_set_repo_report_parser,
_set_repo_sync_parser,
_set_repo_triggers_parser,
]
@@ -1,169 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from collections.abc import Callable
from ahriman.application.application import Application
from ahriman.application.handlers.handler import Handler, SubParserAction
from ahriman.core.configuration import Configuration
from ahriman.core.utils import extract_user
from ahriman.models.packagers import Packagers
from ahriman.models.repository_id import RepositoryId
class Update(Handler):
"""
package update handler
"""
@classmethod
def run(cls, args: argparse.Namespace, repository_id: RepositoryId, configuration: Configuration, *,
report: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
"""
application = Application(repository_id, configuration, report=report, refresh_pacman_database=args.refresh)
application.on_start()
Update.perform_action(application, args)
@staticmethod
def _set_repo_check_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for repository check subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("repo-check", aliases=["check"], help="check for updates",
description="check for packages updates. Same as repo-update --dry-run --no-manual")
parser.add_argument("package", help="filter check by package base", nargs="*")
parser.add_argument("--changes", help="calculate changes from the latest known commit if available",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--check-files", help="enable or disable checking of broken dependencies "
"(e.g. dynamically linked libraries or modules directories)",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty",
action="store_true")
parser.add_argument("--vcs", help="fetch actual version of VCS packages",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("-y", "--refresh", help="download fresh package databases from the mirror before actions, "
"-yy to force refresh even if up to date",
action="count", default=False)
parser.set_defaults(aur=True, dependencies=False, dry_run=True, increment=False, local=True, manual=False,
username=None)
return parser
@staticmethod
def _set_repo_update_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for repository update subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("repo-update", aliases=["update"], help="update packages",
description="check for packages updates and run build process if requested")
parser.add_argument("package", help="filter check by package base", nargs="*")
parser.add_argument("--aur", help="enable or disable checking for AUR updates",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--changes", help="calculate changes from the latest known commit if available. "
"Only applicable in dry run mode",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--check-files", help="enable or disable checking of broken dependencies "
"(e.g. dynamically linked libraries or modules directories)",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--dependencies", help="process missing package dependencies",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--dry-run", help="just perform check for updates, same as check command",
action="store_true")
parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty",
action="store_true")
parser.add_argument("--increment", help="increment package release (pkgrel) on duplicate",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--local", help="enable or disable checking of local packages for updates",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--manual", help="include or exclude manual updates",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("-u", "--username", help="build as user", default=extract_user())
parser.add_argument("--vcs", help="fetch actual version of VCS packages",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("-y", "--refresh", help="download fresh package databases from the mirror before actions, "
"-yy to force refresh even if up to date",
action="count", default=False)
return parser
@staticmethod
def log_fn(application: Application, dry_run: bool) -> Callable[[str], None]:
"""
package updates log function
Args:
application(Application): application instance
dry_run(bool): do not perform update itself
Returns:
Callable[[str], None]: in case if dry_run is set it will return print, logger otherwise
"""
def inner(line: str) -> None:
return print(line) if dry_run else application.logger.info(line) # pylint: disable=bad-builtin
return inner
@staticmethod
def perform_action(application: Application, args: argparse.Namespace) -> None:
"""
perform update action
Args:
application(Application): application instance
args(argparse.Namespace): command line args
"""
packages = application.updates(args.package, aur=args.aur, local=args.local, manual=args.manual, vcs=args.vcs,
check_files=args.check_files)
if args.changes: # generate changes if requested
application.changes(packages)
if args.dry_run: # exit from application if no build requested
Update.check_status(args.exit_code, packages) # status code check
return
packages = application.with_dependencies(packages, process_dependencies=args.dependencies)
packagers = Packagers(args.username, {package.base: package.packager for package in packages})
application.print_updates(packages, log_fn=application.logger.info)
result = application.update(packages, packagers, bump_pkgrel=args.increment)
Update.check_status(args.exit_code, not result.is_empty)
arguments = [
_set_repo_check_parser,
_set_repo_update_parser,
]
@@ -1,169 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
import getpass
from ahriman.application.handlers.handler import Handler, SubParserAction
from ahriman.core.configuration import Configuration
from ahriman.core.database import SQLite
from ahriman.core.exceptions import PasswordError
from ahriman.core.formatters import UserPrinter
from ahriman.core.utils import enum_values
from ahriman.models.action import Action
from ahriman.models.repository_id import RepositoryId
from ahriman.models.user import User
from ahriman.models.user_access import UserAccess
class Users(Handler):
"""
user management handler
"""
ALLOW_MULTI_ARCHITECTURE_RUN = False # system-wide action
@classmethod
def run(cls, args: argparse.Namespace, repository_id: RepositoryId, configuration: Configuration, *,
report: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
"""
database = SQLite.load(configuration)
match args.action:
case Action.Update:
user = Users.user_create(args)
# if password is left blank we are not going to require salt to be set
salt = configuration.get("auth", "salt", fallback="") if user.password else ""
database.user_update(user.hash_password(salt))
case Action.List:
users = database.user_list(args.username, args.role)
for user in users:
UserPrinter(user)(verbose=True)
Users.check_status(args.exit_code, users)
case Action.Remove:
database.user_remove(args.username)
@staticmethod
def _set_user_add_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for create user subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("user-add", help="create or update user",
description="update user for web services with the given password and role. "
"In case if password was not entered it will be asked interactively")
parser.add_argument("username", help="username for web service")
parser.add_argument("--key", help="optional PGP key used by this user. The private key must be imported")
parser.add_argument("--packager", help="optional packager id used for build process in form of "
"`Name Surname <mail@example.com>`")
parser.add_argument(
"-p", "--password", help="user password. Blank password will be treated as empty password, "
"which is in particular must be used for OAuth2 authorization type.")
parser.add_argument("-R", "--role", help="user access level",
type=UserAccess, choices=enum_values(UserAccess), default=UserAccess.Read)
parser.set_defaults(action=Action.Update, architecture="", exit_code=False, lock=None, quiet=True,
report=False, repository="")
return parser
@staticmethod
def _set_user_list_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for user list subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("user-list", help="user known users and their access",
description="list users from the user mapping and their roles")
parser.add_argument("username", help="filter users by username", nargs="?")
parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty",
action="store_true")
parser.add_argument("-R", "--role", help="filter users by role", type=UserAccess,
choices=enum_values(UserAccess))
parser.set_defaults(action=Action.List, architecture="", lock=None, quiet=True, report=False, repository="",
unsafe=True)
return parser
@staticmethod
def _set_user_remove_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for user removal subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("user-remove", help="remove user",
description="remove user from the user mapping and update the configuration")
parser.add_argument("username", help="username for web service")
parser.set_defaults(action=Action.Remove, architecture="", exit_code=False, lock=None, quiet=True,
report=False, repository="")
return parser
@staticmethod
def user_create(args: argparse.Namespace) -> User:
"""
create user descriptor from arguments
Args:
args(argparse.Namespace): command line args
Returns:
User: built user descriptor
Raises:
PasswordError: password input is invalid
"""
def read_password() -> str:
first_password = getpass.getpass()
second_password = getpass.getpass("Repeat password: ")
if first_password != second_password:
raise PasswordError("passwords don't match")
return first_password
password = args.password
if password is None:
password = read_password()
return User(username=args.username, password=password, access=args.role,
packager_id=args.packager, key=args.key)
arguments = [
_set_user_add_parser,
_set_user_list_parser,
_set_user_remove_parser,
]
@@ -1,54 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
import re
import textwrap
class _HelpFormatter(argparse.ArgumentDefaultsHelpFormatter):
"""
:class:`argparse.ArgumentDefaultsHelpFormatter` extension which keeps new lines in help messages
"""
def __init__(self, prog: str) -> None:
"""
Args:
prog(str): application name
"""
argparse.ArgumentDefaultsHelpFormatter.__init__(self, prog, width=120)
self._whitespace_matcher = re.compile(r"[ \t]+")
def _fill_text(self, text: str, width: int, indent: str) -> str:
"""
implementation of the protected method to format text. Format text, keeping new lines
Args:
text(str): text to format
width(int): maximal text width
indent(str): indentation string
Returns:
str: formatted text
"""
text = self._whitespace_matcher.sub(" ", text).strip()
return "\n".join([
textwrap.fill(line, width, initial_indent=indent, subsequent_indent=indent)
for line in text.splitlines()
])
@@ -1,60 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from code import InteractiveConsole
from importlib.util import find_spec
from typing import Any
class InteractiveShell(InteractiveConsole):
"""
wrapper around :class:`code.InteractiveConsole` to pass :func:`interact()` to IPython shell
"""
@staticmethod
def has_ipython() -> bool:
"""
check if IPython shell is available
Returns:
bool: ``True`` if IPython shell is available, ``False`` otherwise
"""
try:
return find_spec("IPython.terminal.embed") is not None
except ModuleNotFoundError:
return False
def interact(self, *args: Any, **kwargs: Any) -> None:
"""
pass controller to IPython shell
Args:
*args(Any): positional arguments
**kwargs(Any): keyword arguments
"""
if self.has_ipython():
from IPython.terminal.embed import InteractiveShellEmbed
shell = InteractiveShellEmbed(user_ns=self.locals) # type: ignore[no-untyped-call]
shell.show_banner() # type: ignore[no-untyped-call]
shell.interact() # type: ignore[no-untyped-call]
else:
# fallback to default
import readline # pylint: disable=unused-import
InteractiveConsole.interact(self, *args, **kwargs)
@@ -1,275 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import itertools
import shutil
import tarfile
from collections.abc import Iterable, Iterator
from functools import cached_property
from pathlib import Path
from pyalpm import DB, Package, SIG_DATABASE_OPTIONAL, SIG_PACKAGE_OPTIONAL # type: ignore[import-not-found]
from string import Template
from ahriman.core.alpm.pacman_database import PacmanDatabase
from ahriman.core.alpm.pacman_handle import PacmanHandle
from ahriman.core.configuration import Configuration
from ahriman.core.log import LazyLogging
from ahriman.core.utils import trim_package
from ahriman.models.pacman_synchronization import PacmanSynchronization
from ahriman.models.repository_id import RepositoryId
class Pacman(LazyLogging):
"""
alpm wrapper
Attributes:
configuration(Configuration): configuration instance
refresh_database(PacmanSynchronization): synchronize local cache to remote
repository_id(RepositoryId): repository unique identifier
repository_paths(RepositoryPaths): repository paths instance
"""
def __init__(self, repository_id: RepositoryId, configuration: Configuration, *,
refresh_database: PacmanSynchronization) -> None:
"""
Args:
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
refresh_database(PacmanSynchronization): synchronize local cache to remote
"""
self.configuration = configuration
self.repository_id = repository_id
self.repository_paths = configuration.repository_paths
self.refresh_database = refresh_database
@cached_property
def handle(self) -> PacmanHandle:
"""
pyalpm handle
Returns:
PacmanHandle: generated pyalpm handle instance
"""
return self.__create_handle(refresh_database=self.refresh_database)
def __create_handle(self, *, refresh_database: PacmanSynchronization) -> PacmanHandle:
"""
create lazy handle function
Args:
refresh_database(PacmanSynchronization): synchronize local cache to remote
Returns:
PacmanHandle: fully initialized pacman handle
"""
pacman_root = self.configuration.getpath("alpm", "database")
use_ahriman_cache = self.configuration.getboolean("alpm", "use_ahriman_cache")
database_path = self.repository_paths.pacman if use_ahriman_cache else pacman_root
root = self.configuration.getpath("alpm", "root")
handle = PacmanHandle(str(root), str(database_path))
for repository in self.configuration.getlist("alpm", "repositories"):
database = self.database_init(handle, repository, self.repository_id.architecture)
self.database_copy(handle, database, pacman_root, use_ahriman_cache=use_ahriman_cache)
# install repository database too (without copying)
self.database_init(handle, self.repository_id.name, self.repository_id.architecture)
if use_ahriman_cache and refresh_database:
self.database_sync(handle, force=refresh_database == PacmanSynchronization.Force)
return handle
def database_copy(self, handle: PacmanHandle, database: DB, pacman_root: Path, *, use_ahriman_cache: bool) -> None:
"""
copy database from the operating system root to the ahriman local home
Args:
handle(PacmanHandle): pacman handle which will be used for database copying
database(DB): pacman database instance to be copied
pacman_root(Path): operating system pacman root
use_ahriman_cache(bool): use local ahriman cache instead of system one
"""
def repository_database(root: Path) -> Path:
return root / "sync" / f"{database.name}.db"
if not use_ahriman_cache:
return
# copy root database if no local copy found
pacman_db_path = Path(handle.dbpath)
if not pacman_db_path.is_dir():
return # root directory does not exist yet
dst = repository_database(pacman_db_path)
if dst.is_file():
return # file already exists, do not copy
dst.parent.mkdir(mode=0o755, exist_ok=True) # create sync directory if it doesn't exist
src = repository_database(pacman_root)
if not src.is_file():
self.logger.warning("repository %s is set to be used, however, no working copy was found", database.name)
return # database for some reason deos not exist
self.logger.info("copy pacman database %s from operating system root to ahriman's home %s", src, dst)
with self.repository_paths.preserve_owner():
shutil.copy(src, dst)
def database_init(self, handle: PacmanHandle, repository: str, architecture: str) -> DB:
"""
create database instance from pacman handler and set its properties
Args:
handle(PacmanHandle): pacman handle which will be used for database initializing
repository(str): pacman repository name (e.g. core)
architecture(str): repository architecture
Returns:
DB: loaded pacman database instance
"""
self.logger.info("loading pacman database %s", repository)
database: DB = handle.register_syncdb(repository, SIG_DATABASE_OPTIONAL | SIG_PACKAGE_OPTIONAL)
if repository != self.repository_id.name:
mirror = self.configuration.get("alpm", "mirror")
# replace variables in mirror address
variables = {
"arch": architecture,
"repo": repository,
}
server = Template(mirror).safe_substitute(variables)
else:
# special case, same database, use local storage instead
server = f"file://{self.repository_paths.repository}"
database.servers = [server]
return database
def database_sync(self, handle: PacmanHandle, *, force: bool) -> None:
"""
sync local database
Args:
handle(PacmanHandle): pacman handle which will be used for database sync
force(bool): force database synchronization (same as ``pacman -Syy``)
"""
self.logger.info("refresh ahriman's home pacman database (force refresh %s)", force)
transaction = handle.init_transaction()
for database in handle.get_syncdbs():
PacmanDatabase(database, self.configuration).sync(force=force)
transaction.release()
def files(self, packages: Iterable[str]) -> dict[str, set[str]]:
"""
extract list of known packages from the databases
Args:
packages(Iterable[str]): filter by package names
Returns:
dict[str, set[str]]: map of package name to its list of files
"""
def extract(tar: tarfile.TarFile, versions: dict[str, str]) -> Iterator[tuple[str, set[str]]]:
for package_name, version in versions.items():
path = Path(f"{package_name}-{version}") / "files"
try:
content = tar.extractfile(str(path))
except KeyError:
# in case if database and its files has been desync somehow, the extractfile will raise
# KeyError because the entry doesn't exist
content = None
if content is None:
continue
# this is just array of files, however, the directories are with trailing slash,
# which previously has been removed by the conversion to ``pathlib.Path``
files = {filename.decode("utf8").rstrip().removesuffix("/") for filename in content.readlines()}
yield package_name, files
# sort is required for the following group by operation
descriptors = sorted(
(package for package_name in packages for package in self.package(package_name)),
key=lambda package: package.db.name
)
result: dict[str, set[str]] = {}
for database_name, pacman_packages in itertools.groupby(descriptors, lambda package: package.db.name):
database_file = self.repository_paths.pacman / "sync" / f"{database_name}.files.tar.gz"
if not database_file.is_file():
continue # no database file found
package_names = {package.name: package.version for package in pacman_packages}
with tarfile.open(database_file, "r:gz") as archive:
result.update(extract(archive, package_names))
return result
def package(self, package_name: str) -> Iterator[Package]:
"""
retrieve list of the packages from the repository by name
Args:
package_name(str): package name to search
Yields:
Package: list of packages which were returned by the query
"""
for database in self.handle.get_syncdbs():
package = database.get_pkg(package_name)
if package is None:
continue
yield package
def packages(self) -> set[str]:
"""
get list of packages known for alpm
Returns:
set[str]: list of package names
"""
result: set[str] = set()
for database in self.handle.get_syncdbs():
for package in database.pkgcache:
# package itself
result.add(package.name)
# provides list for meta-packages
result.update(trim_package(provides) for provides in package.provides)
return result
def provided_by(self, package_name: str) -> Iterator[Package]:
"""
search through databases and emit packages which provides the ``package_name``
Args:
package_name(str): package name to search
Yields:
Package: list of packages which were returned by the query
"""
def is_package_provided(package: Package) -> bool:
provides = [trim_package(name) for name in package.provides]
return package_name in provides
for database in self.handle.get_syncdbs():
yield from filter(is_package_provided, database.search(package_name))
@@ -1,176 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import os
import shutil
from email.utils import parsedate_to_datetime
from pathlib import Path
from pyalpm import DB # type: ignore[import-not-found]
from typing import ClassVar
from urllib.parse import urlparse
from ahriman.core.configuration import Configuration
from ahriman.core.exceptions import PacmanError
from ahriman.core.http import SyncHttpClient
class PacmanDatabase(SyncHttpClient):
"""
implementation for database sync, because pyalpm is not always enough
Attributes:
LAST_MODIFIED_HEADER(str): last modified header name
database(DB): pyalpm database object
repository_paths(RepositoryPaths): repository paths instance
sync_files_database(bool): sync files database
"""
LAST_MODIFIED_HEADER: ClassVar[str] = "Last-Modified"
def __init__(self, database: DB, configuration: Configuration) -> None:
"""
Args:
database(DB): pyalpm database object
configuration(Configuration): configuration instance
"""
SyncHttpClient.__init__(self)
self.timeout = None # reset timeout
self.database = database
self.repository_paths = configuration.repository_paths
self.sync_files_database = configuration.getboolean("alpm", "sync_files_database")
@staticmethod
def copy(remote_path: Path, local_path: Path) -> None:
"""
copy local database file
Args:
remote_path(Path): path to source (remote) file
local_path(Path): path to locally stored file
"""
shutil.copy(remote_path, local_path)
def download(self, url: str, local_path: Path) -> None:
"""
download remote file and store it to local path with the correct last modified headers
Args:
url(str): remote url to request file
local_path(Path): path to locally stored file
Raises:
PacmanError: in case if no last-modified header was found
"""
response = self.make_request("GET", url, stream=True)
if self.LAST_MODIFIED_HEADER not in response.headers:
raise PacmanError("No last-modified header found")
with local_path.open("wb") as local_file:
for chunk in response.iter_content(chunk_size=1024):
local_file.write(chunk)
# set correct (a,m)time for the file
remote_changed = parsedate_to_datetime(response.headers[self.LAST_MODIFIED_HEADER]).timestamp()
os.utime(local_path, (remote_changed, remote_changed))
def is_outdated(self, url: str, local_path: Path) -> bool:
"""
check if local file is outdated
Args:
url(str): remote url to request last modified header
local_path(Path): path to locally stored file
Returns:
bool: ``True`` in case if remote file is newer than local file
Raises:
PacmanError: in case if no last-modified header was found
"""
if not local_path.is_file():
return True # no local file found, requires to update
response = self.make_request("HEAD", url)
if self.LAST_MODIFIED_HEADER not in response.headers:
raise PacmanError("No last-modified header found")
remote_changed = parsedate_to_datetime(response.headers["Last-Modified"]).timestamp()
local_changed = local_path.stat().st_mtime
return remote_changed > local_changed
def sync(self, *, force: bool) -> None:
"""
sync packages and files databases
Args:
force(bool): force database synchronization (same as ``pacman -Syy``)
"""
try:
self.sync_packages(force=force)
if self.sync_files_database:
self.sync_files(force=force)
except Exception:
self.logger.exception("exception during update %s", self.database.name)
def sync_files(self, *, force: bool) -> None:
"""
sync files by using http request
Args:
force(bool): force database synchronization (same as ``pacman -Syy``)
Raises:
PacmanError: on operation error (invalid scheme or incomplete configuration)
"""
try:
server = next(iter(self.database.servers))
except StopIteration:
raise PacmanError("No configured servers available for database") from None
filename = f"{self.database.name}.files.tar.gz"
url = f"{server}/{filename}"
remote_uri = urlparse(url)
local_path = Path(self.repository_paths.pacman / "sync" / filename)
match remote_uri.scheme:
case "http" | "https":
if not force and not self.is_outdated(url, local_path):
return
self.download(url, local_path)
case "file":
# just copy file as it is relatively cheap operation, no need to check timestamps
self.copy(Path(remote_uri.path), local_path)
case other:
raise PacmanError(f"Unknown or unsupported URL scheme {other}")
def sync_packages(self, *, force: bool) -> None:
"""
sync packages by using built-in pyalpm methods
Args:
force(bool): force database synchronization (same as ``pacman -Syy``)
"""
self.database.update(force)
@@ -1,81 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from pathlib import Path
from pyalpm import Handle, Package # type: ignore[import-not-found]
from tempfile import TemporaryDirectory
from typing import Any, ClassVar, Self
class PacmanHandle:
"""
lightweight wrapper for pacman handle to be used for direct alpm operations (e.g. package load)
Attributes:
handle(Handle): pyalpm handle instance
"""
_ephemeral: ClassVar[Self | None] = None
def __init__(self, *args: Any, **kwargs: Any) -> None:
"""
Args:
*args(Any): positional arguments for :class:`pyalpm.Handle`
**kwargs(Any): keyword arguments for :class:`pyalpm.Handle`
"""
self.handle = Handle(*args, **kwargs)
@classmethod
def ephemeral(cls) -> Self:
"""
create temporary instance with no access to real databases
Returns:
Self: loaded class
"""
if cls._ephemeral is None:
# handle creates alpm version file, but we don't use it
# so it is ok to just remove it
with TemporaryDirectory(ignore_cleanup_errors=True) as dir_name:
cls._ephemeral = cls("/", dir_name)
return cls._ephemeral
def package_load(self, path: Path) -> Package:
"""
load package from path to the archive
Args:
path(Path): path to package archive
Returns:
Package: package instance
"""
return self.handle.load_pkg(str(path))
def __getattr__(self, item: str) -> Any:
"""
proxy methods for :class:`pyalpm.Handle`, because it doesn't allow subclassing
Args:
item(str): property name
Returns:
Any: attribute by its name
"""
return self.handle.__getattribute__(item)
@@ -1,371 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import itertools
import re
import shlex
from collections.abc import Iterator
from enum import StrEnum
from typing import IO
from ahriman.core.exceptions import PkgbuildParserError
from ahriman.models.pkgbuild_patch import PkgbuildPatch
class PkgbuildToken(StrEnum):
"""
well-known tokens dictionary
Attributes:
ArrayEnds(PkgbuildToken): array ends token
ArrayStarts(PkgbuildToken): array starts token
Comma(PkgbuildToken): comma token
Comment(PkgbuildToken): comment token
FunctionDeclaration(PkgbuildToken): function declaration token
FunctionEnds(PkgbuildToken): function ends token
FunctionStarts(PkgbuildToken): function starts token
NewLine(PkgbuildToken): new line token
"""
ArrayStarts = "("
ArrayEnds = ")"
Comma = ","
Comment = "#"
FunctionDeclaration = "()"
FunctionStarts = "{"
FunctionEnds = "}"
NewLine = "\n"
class PkgbuildParser(shlex.shlex):
"""
simple pkgbuild reader implementation in pure python, because others suck.
What is it:
#. Simple PKGBUILD parser written in python.
#. No shell execution, so it is free from random shell attacks.
#. Able to parse simple constructions (assignments, comments, functions, arrays).
What it is not:
#. Fully functional shell parser.
#. Shell executor.
#. No parameter expansion.
For more details what does it support, please, consult with the test cases.
Examples:
This class is heavily based on :mod:`shlex` parser, but instead of strings operates with the
:class:`ahriman.models.pkgbuild_patch.PkgbuildPatch` objects. The main way to use it is to call :func:`parse()`
function and collect parsed objects, e.g.::
>>> parser = PkgbuildParser(StringIO("input string"))
>>> for patch in parser.parse():
>>> print(f"{patch.key} = {patch.value}")
It doesn't store the state of the fields (but operates with the :mod:`shlex` parser state), so no shell
post-processing is performed (e.g. variable substitution).
"""
_ARRAY_ASSIGNMENT = re.compile(r"^(?P<key>\w+)=$")
# in addition to usual assignment, functions can have dash
_FUNCTION_DECLARATION = re.compile(r"^(?P<key>[\w-]+)$")
_STRING_ASSIGNMENT = re.compile(r"^(?P<key>\w+)=(?P<value>.+)$")
def __init__(self, stream: IO[str]) -> None:
"""
Args:
stream(IO[str]): input stream containing PKGBUILD content
"""
shlex.shlex.__init__(self, stream, posix=True, punctuation_chars=True)
self._io = stream # direct access without type casting
# ignore substitution and extend bash symbols
self.wordchars += "${}#:+-@!"
# in case of default behavior, it will ignore, for example, segment part of url outside of quotes
self.commenters = ""
@staticmethod
def _expand_array(array: list[str]) -> list[str]:
"""
bash array expansion simulator. It takes raw array and tries to expand constructions like
``(first prefix-{mid1,mid2}-suffix last)`` into ``(first, prefix-mid1-suffix prefix-mid2-suffix last)``
Args:
array(list[str]): input array
Returns:
list[str]: either source array or expanded array if possible
Raises:
PkgbuildParserError: if there are errors in parser
"""
# we are using comma as marker for expansion (if any)
if PkgbuildToken.Comma not in array:
return array
# again sanity check, for expansion there are at least 3 elements (first, last and comma)
if len(array) < 3:
return array
result = []
buffer, prefix = [], None
for index, (first, second) in enumerate(itertools.pairwise(array)):
match (first, second):
# in this case we check if expansion should be started
# this condition matches "prefix{first", ","
case (_, PkgbuildToken.Comma) if PkgbuildToken.FunctionStarts in first:
prefix, part = first.rsplit(PkgbuildToken.FunctionStarts, maxsplit=1)
buffer.append(f"{prefix}{part}")
# the last element case, it matches either ",", "last}" or ",", "last}suffix"
# in case if there is suffix, it must be appended to all list elements
case (PkgbuildToken.Comma, _) if prefix is not None and PkgbuildToken.FunctionEnds in second:
part, suffix = second.rsplit(PkgbuildToken.FunctionEnds, maxsplit=1)
buffer.append(f"{prefix}{part}")
result.extend([f"{part}{suffix}" for part in buffer])
# reset state
buffer, prefix = [], None
# we have already got prefix string, so we are in progress of expansion
# we always operate the last element, so this matches ",", "next"
case (PkgbuildToken.Comma, _) if prefix is not None:
buffer.append(f"{prefix}{second}")
# exactly first element of the list
case (_, _) if prefix is None and index == 0:
result.append(first)
# any next normal element
case (_, _) if prefix is None:
result.append(second)
# small sanity check
if prefix is not None:
raise PkgbuildParserError("error in array expansion", array)
return result
def _is_escaped(self) -> bool:
"""
check if the last element was quoted. ``shlex.shlex`` parser doesn't provide information about was the token
quoted or not, thus there is no difference between "'#'" (sharp in quotes) and "#" (sharp without quotes). This
method simply rolls back to the last non-space character and check if it is a quotation mark
Returns:
bool: ``True`` if the previous element of the stream is a quote or escaped and ``False`` otherwise
"""
current_position = self._io.tell()
last_char = penultimate_char = None
index = current_position - 1
while index > 0:
index, last_char = self._read_last(index)
if last_char.isspace():
index -= 1
continue
if index > 1:
_, penultimate_char = self._read_last(index - 1)
break
self._io.seek(current_position) # reset position of the stream
is_quoted = last_char is not None and last_char in self.quotes
is_escaped = penultimate_char is not None and penultimate_char in self.escape
return is_quoted or is_escaped
def _parse_array(self) -> list[str]:
"""
parse array from the PKGBUILD. This method will extract tokens from parser until it matches closing array,
modifying source parser state
Returns:
list[str]: extracted arrays elements
Raises:
PkgbuildParserError: if array is not closed
"""
def extract() -> Iterator[str]:
while token := self.get_token():
match token:
case _ if self._is_escaped():
pass
case PkgbuildToken.ArrayEnds:
break
case comment if comment.startswith(PkgbuildToken.Comment):
self._read_comment()
continue
yield token
if token != PkgbuildToken.ArrayEnds:
raise PkgbuildParserError("no closing array bracket found")
return self._expand_array(list(extract()))
def _parse_function(self) -> str:
"""
parse function from the PKGBUILD. This method will extract tokens from parser until it matches closing function,
modifying source parser state. Instead of trying to combine tokens together, it uses positions of the file
and reads content again in this range
Returns:
str: function body
Raises:
PkgbuildParserError: if function body wasn't found or parser input stream doesn't support position reading
"""
# find start and end positions
start_position = end_position = -1
counter = 0 # simple processing of the inner "{" and "}"
for token in self:
match token:
case _ if self._is_escaped():
continue
case PkgbuildToken.FunctionStarts:
if counter == 0:
start_position = self._io.tell() - 1
counter += 1
case PkgbuildToken.FunctionEnds:
end_position = self._io.tell()
if self.state != self.eof: # type: ignore[attr-defined]
end_position -= 1 # if we are not at the end of the file, position is _after_ the token
counter -= 1
if counter == 0:
break
case comment if comment.startswith(PkgbuildToken.Comment):
self._read_comment()
if not 0 < start_position < end_position:
raise PkgbuildParserError("function body wasn't found")
# read the specified interval from source stream
self._io.seek(start_position - 1) # start from the previous symbol
# we cannot use :func:`read()` here, because it reads characters, not bytes
content = ""
while self._io.tell() != end_position and (next_char := self._io.read(1)):
content += next_char
# special case of the end of file
if self.state == self.eof: # type: ignore[attr-defined]
content += self._io.read(1)
return content
def _parse_token(self, token: str) -> Iterator[PkgbuildPatch]:
"""
parse single token to the PKGBUILD field
Args:
token(str): current token
Yields:
PkgbuildPatch: extracted a PKGBUILD node
"""
# simple assignment rule
if m := self._STRING_ASSIGNMENT.match(token):
key = m.group("key")
value = m.group("value")
yield PkgbuildPatch(key, value)
return
if token.startswith(PkgbuildToken.Comment):
self._read_comment()
return
match self.get_token():
# array processing. Arrays will be sent as "key=", "(", values, ")"
case PkgbuildToken.ArrayStarts if m := self._ARRAY_ASSIGNMENT.match(token):
key = m.group("key")
value = self._parse_array()
yield PkgbuildPatch(key, value)
# functions processing. Function will be sent as "name", "()", "{", body, "}"
case PkgbuildToken.FunctionDeclaration if self._FUNCTION_DECLARATION.match(token):
key = f"{token}{PkgbuildToken.FunctionDeclaration}"
value = self._parse_function()
yield PkgbuildPatch(key, value) # this is not mistake, assign to token without ()
# special function case, where "(" and ")" are separated tokens, e.g. "pkgver ( )"
case PkgbuildToken.ArrayStarts if self._FUNCTION_DECLARATION.match(token):
next_token = self.get_token()
if next_token == PkgbuildToken.ArrayEnds: # replace closing bracket with "()"
next_token = PkgbuildToken.FunctionDeclaration
self.push_token(next_token) # type: ignore[arg-type]
yield from self._parse_token(token)
# some random token received without continuation, lets guess it is empty assignment (i.e. key=)
case other if other is not None:
yield from self._parse_token(other)
def _read_comment(self) -> None:
"""
read comment from the current position. This method doesn't check comment itself, just read the stream
until the comment line ends
"""
_, last_symbol = self._read_last()
if last_symbol != PkgbuildToken.NewLine:
self.instream.readline()
def _read_last(self, initial_index: int | None = None) -> tuple[int, str]:
"""
wrapper around read to read the last symbol from the input stream. This method is designed to process UTF-8
symbols correctly. This method does not reset current stream position
Args:
initial_index(int | None, optional): initial index to start reading from. If none set, the previous position
will be used (Default value = None)
Returns:
tuple[int, str]: last symbol and its position in the stream
Raises:
PkgbuildParserError: in case if stream reached starting position, but no valid symbols were found
"""
if initial_index is None:
initial_index = self._io.tell() - 1
if initial_index < 0:
raise PkgbuildParserError("stream is on starting position")
self._io.seek(initial_index)
while (position := self._io.tell()) > 0:
try:
return position, self._io.read(1)
except UnicodeDecodeError:
self._io.seek(position - 1)
raise PkgbuildParserError("reached starting position, no valid symbols found")
def parse(self) -> Iterator[PkgbuildPatch]:
"""
parse source stream and yield parsed entries
Yields:
PkgbuildPatch: extracted a PKGBUILD node
"""
for token in self:
yield from self._parse_token(token)
@@ -1,263 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from elftools.elf.dynamic import DynamicSection
from elftools.elf.elffile import ELFFile
from pathlib import Path
from typing import IO
from ahriman.core.alpm.pacman import Pacman
from ahriman.core.alpm.remote import OfficialSyncdb
from ahriman.core.exceptions import UnknownPackageError
from ahriman.core.utils import walk
from ahriman.models.dependencies import Dependencies
from ahriman.models.filesystem_package import FilesystemPackage
from ahriman.models.package import Package
from ahriman.models.scan_paths import ScanPaths
class PackageArchive:
"""
helper for package archives
Attributes:
package(Package): package descriptor
pacman(Pacman): alpm wrapper instance
root(Path): path to root filesystem
scan_paths(ScanPaths): scan paths holder
"""
def __init__(self, root: Path, package: Package, pacman: Pacman, scan_paths: ScanPaths) -> None:
"""
Args:
root(Path): path to root filesystem
package(Package): package descriptor
pacman(Pacman): alpm wrapper instance
scan_paths(ScanPaths): scan paths holder
"""
self.root = root
self.package = package
self.pacman = pacman
self.scan_paths = scan_paths
@staticmethod
def dynamic_needed(binary_path: Path) -> list[str]:
"""
extract dynamic libraries required by the specified file
Args:
binary_path(Path): path to library, file, etc
Returns:
list[str]: libraries which this file linked dynamically. Returns empty set in case if file is not
a binary or no dynamic section has been found
"""
with binary_path.open("rb") as binary_file:
if not PackageArchive.is_elf(binary_file):
return []
elf_file = ELFFile(binary_file)
dynamic_section = next(
(section for section in elf_file.iter_sections()
if isinstance(section, DynamicSection)),
None)
if dynamic_section is None:
return []
return [
tag.needed # type: ignore[attr-defined]
for tag in dynamic_section.iter_tags()
if tag.entry.d_tag == "DT_NEEDED"
]
@staticmethod
def is_elf(content: IO[bytes]) -> bool:
"""
check if the content is actually elf file
Args:
content(IO[bytes]): content of the file
Returns:
bool: ``True`` in case if file has elf header and ``False`` otherwise
"""
expected = b"\x7fELF"
length = len(expected)
magic_bytes = content.read(length)
content.seek(0) # reset reading position
return magic_bytes == expected
def _load_pacman_package(self, path: Path) -> FilesystemPackage:
"""
load pacman package model from path
Args:
path(Path): path to package files database
Returns:
FilesystemPackage: generated pacman package model with empty paths
"""
package_name, *_ = path.parent.name.rsplit("-", maxsplit=2)
try:
pacman_package = OfficialSyncdb.info(package_name, pacman=self.pacman)
return FilesystemPackage(
package_name=package_name,
depends=set(pacman_package.depends),
opt_depends=set(pacman_package.opt_depends),
)
except UnknownPackageError:
return FilesystemPackage(package_name=package_name, depends=set(), opt_depends=set())
def _raw_dependencies_packages(self) -> dict[Path, list[FilesystemPackage]]:
"""
extract the initial list of packages which contain specific path this package depends on
Returns:
dict[Path, list[FilesystemPackage]]: map of path to packages containing this path
"""
dependencies, roots = self.depends_on_paths()
installed_packages = self.installed_packages()
# build list of packages, which contains both the package itself and (possible) debug packages
packages = list(self.package.packages) + [f"{self.package.base}-debug"]
# build initial map of file path -> packages containing this path
# in fact, keys will contain all libraries the package linked to and all directories it contains
dependencies_per_path: dict[Path, list[FilesystemPackage]] = {}
for package_base, package in installed_packages.items():
if package_base in packages:
continue # skip package itself
required_by = [directory for directory in package.directories if directory in roots]
required_by.extend(library for library in package.files if library.name in dependencies)
for path in required_by:
dependencies_per_path.setdefault(path, []).append(package)
return dependencies_per_path
def _refine_dependencies(self, source: dict[Path, list[FilesystemPackage]]) -> dict[Path, list[FilesystemPackage]]:
"""
reduce the initial dependency list by removing packages which are already satisfied (e.g. by other path or by
dependency list, or belonging to the base packages)
Args:
source(dict[Path, list[FilesystemPackage]]): the initial map of path to packages containing it
Returns:
dict[Path, list[FilesystemPackage]]: reduced source map of packages
"""
# base packages should be always excluded from checking
base_packages = OfficialSyncdb.info("base", pacman=self.pacman).depends
result: dict[Path, list[FilesystemPackage]] = {}
# sort items from children directories to root
for path, packages in sorted(source.items(), reverse=True):
# skip if this path belongs to the one of the base packages
if any(package.package_name in base_packages for package in packages):
continue
# check path against the black/white listed
if not self.scan_paths.is_allowed(path):
continue
# remove explicit dependencies
packages = [package for package in packages if package.is_root_package(packages, include_optional=False)]
# remove optional dependencies
packages = [package for package in packages if package.is_root_package(packages, include_optional=True)]
# check if there is already parent of current path in the result and has the same packages
for children_path, children_packages in result.items():
if not children_path.is_relative_to(path):
continue
children_packages_names = {package.package_name for package in children_packages}
packages = [package for package in packages if package.package_name not in children_packages_names]
result[path] = packages
return result
def depends_on(self) -> Dependencies:
"""
extract packages and paths which are required for this package
Returns:
Dependencies: map of the package name to set of paths used by this package
"""
initial_packages = self._raw_dependencies_packages()
refined_packages = self._refine_dependencies(initial_packages)
paths = {
str(path): [package.package_name for package in packages]
for path, packages in refined_packages.items()
}
return Dependencies(paths)
def depends_on_paths(self) -> tuple[set[str], set[Path]]:
"""
extract dependencies from installation
Returns:
tuple[set[str], set[Path]]: tuple of dynamically linked libraries and directory paths
"""
dependencies = set()
roots: set[Path] = set()
for package in self.package.packages:
package_dir = self.root / "build" / self.package.base / "pkg" / package
for path in filter(lambda p: p.is_file(), walk(package_dir)):
dependencies.update(PackageArchive.dynamic_needed(path))
filesystem_path = Path(*path.relative_to(package_dir).parts)
roots.update(filesystem_path.parents[:-1]) # last element is always . because paths are relative
return dependencies, roots
def installed_packages(self) -> dict[str, FilesystemPackage]:
"""
extract list of the installed packages and their content
Returns:
dict[str, FilesystemPackage]: map of package name to list of directories and files contained
by this package
"""
result = {}
pacman_local_files = self.root / "var" / "lib" / "pacman" / "local"
for path in filter(lambda fn: fn.name == "files", walk(pacman_local_files)):
package = self._load_pacman_package(path)
is_files_section = False
for line in path.read_text(encoding="utf8").splitlines():
if not line: # skip empty lines
continue
if line.startswith("%") and line.endswith("%"): # directive started
is_files_section = line == "%FILES%"
if not is_files_section: # not a files directive
continue
entry = Path(line)
if line.endswith("/"): # simple check if it is directory
package.directories.append(entry)
else:
package.files.append(entry)
result[package.package_name] = package
return result
@@ -1,114 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from ahriman.core.build_tools.task import Task
from ahriman.core.configuration import Configuration
from ahriman.core.log import LazyLogging
from ahriman.core.utils import full_version, utcnow
from ahriman.models.package import Package
from ahriman.models.pkgbuild import Pkgbuild
class PackageVersion(LazyLogging):
"""
package version extractor and helper
Attributes:
package(Package): package definitions
"""
def __init__(self, package: Package) -> None:
"""
Args:
package(Package): package definitions
"""
self.package = package
def actual_version(self, configuration: Configuration) -> str:
"""
additional method to handle VCS package versions
Args:
configuration(Configuration): configuration instance
Returns:
str: package version if package is not VCS and current version according to VCS otherwise
"""
if not self.package.is_vcs:
return self.package.version
_, repository_id = configuration.check_loaded()
paths = configuration.repository_paths
task = Task(self.package, configuration, repository_id.architecture, paths)
try:
# create fresh chroot environment, fetch sources and - automagically - update PKGBUILD
task.init(paths.cache_for(self.package.base), [], None)
pkgbuild = Pkgbuild.from_file(paths.cache_for(self.package.base) / "PKGBUILD")
return full_version(pkgbuild.get("epoch"), pkgbuild["pkgver"], pkgbuild["pkgrel"])
except Exception:
self.logger.exception("cannot determine version of VCS package")
finally:
# clear log files generated by devtools
for log_file in paths.cache_for(self.package.base).glob("*.log"):
log_file.unlink()
return self.package.version
def is_newer_than(self, timestamp: float | int) -> bool:
"""
check if package was built after the specified timestamp
Args:
timestamp(float | int): timestamp to check build date against
Returns:
bool: ``True`` in case if package was built after the specified date and ``False`` otherwise.
In case if build date is not set by any of packages, it returns False
"""
return any(
package.build_date > timestamp
for package in self.package.packages.values()
if package.build_date is not None
)
def is_outdated(self, remote: Package, configuration: Configuration, *,
calculate_version: bool = True) -> bool:
"""
check if package is out-of-dated
Args:
remote(Package): package properties from remote source
configuration(Configuration): configuration instance
calculate_version(bool, optional): expand version to actual value (by calculating git versions)
(Default value = True)
Returns:
bool: ``True`` if the package is out-of-dated and ``False`` otherwise
"""
vcs_allowed_age = configuration.getint("build", "vcs_allowed_age", fallback=0)
min_vcs_build_date = utcnow().timestamp() - vcs_allowed_age
if calculate_version and not self.is_newer_than(min_vcs_build_date):
remote_version = PackageVersion(remote).actual_version(configuration)
else:
remote_version = remote.version
return self.package.vercmp(remote_version) < 0
@@ -1,91 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from typing import Any
from ahriman.core.exceptions import OptionError
class ConfigurationMultiDict(dict[str, Any]):
"""
wrapper around :class:`dict` to handle multiple configuration keys as lists if they end with ``[]``.
Examples:
This class is designed to be used only with :class:`configparser.RawConfigParser` class, but idea is that
if the key ends with ``[]`` it will be treated as array and the result will be appended to the current value.
In addition, if the value is empty, then it will clear previous values, e.g.:
>>> data = ConfigurationMultiDict()
>>>
>>> data["single"] = "value" # append normal key
>>> print(data) # {"single": "value"}
>>>
>>> data["array[]"] = ["value1"] # append array value
>>> data["array[]"] = ["value2"]
>>> print(data) # {"single": "value", "array": ["value1 value2"]}
>>>
>>> data["array[]"] = [""] # clear previous values
>>> data["array[]"] = ["value3"]
>>> print(data) # {"single": "value", "array": ["value3"]}
"""
def _set_array_value(self, key: str, value: Any) -> None:
"""
set array value. If the key already exists in the dictionary, its value will be prepended to new value
Args:
key(str): key to insert
value(Any): value of the related key
Raises:
OptionError: if the key already exists in the dictionary, but not a single value list or a string
"""
match self.get(key):
case [current_value] | (str() as current_value):
value = f"{current_value} {value}"
case None:
pass
case other:
raise OptionError(other)
super().__setitem__(key, [value])
def __setitem__(self, key: str, value: Any) -> None:
"""
set ``key`` to ``value``. If the value equals to ``[""]`` (array with empty string), then the key
will be removed (as expected from :class:`configparser.RawConfigParser`). If the key ends with
``[]``, the value will be treated as an array and vice versa.
Args:
key(str): key to insert
value(Any): value of the related key
Raises:
OptionError: if ``key`` contains ``[]``, but not at the end of the string (e.g. ``prefix[]suffix``)
"""
real_key, is_key_array, remaining = key.partition("[]")
if remaining:
raise OptionError(key)
match value:
case [""]: # empty value key
self.pop(real_key, None)
case [array_value] if is_key_array: # update array value
self._set_array_value(real_key, array_value)
case _: # normal key
super().__setitem__(real_key, value)
@@ -1,122 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import configparser
import os
import sys
from collections.abc import Iterator, Mapping, MutableMapping
from string import Template
from typing import Any, ClassVar
from ahriman.core.configuration.shell_template import ShellTemplate
class ShellInterpolator(configparser.Interpolation):
"""
custom string interpolator, because we cannot use defaults argument due to config validation
"""
DATA_LINK_ESCAPE: ClassVar[str] = "\x10"
@staticmethod
def _extract_variables(parser: MutableMapping[str, Mapping[str, str]], value: str,
defaults: Mapping[str, str]) -> Iterator[tuple[str, str]]:
"""
extract keys and values (if available) from the configuration. In case if a key is not available, it will be
silently skipped from the result
Args:
parser(MutableMapping[str, Mapping[str, str]]): option parser
value(str): source (not-converted) value
defaults(Mapping[str, str]): default values
Yields:
tuple[str, str]: variable name used for substitution and its value
"""
def identifiers() -> Iterator[tuple[str | None, str]]:
# extract all found identifiers and parse them
for identifier in ShellTemplate(value).get_identifiers():
match identifier.rsplit(":", maxsplit=1):
case [lookup_option]: # single option from the same section
yield None, lookup_option
case [lookup_section, lookup_option]: # reference to another section
yield lookup_section, lookup_option
for section, option in identifiers():
# key to be substituted
key = f"{section}:{option}" if section else option
if section is not None: # foreign section case
if section not in parser:
continue # section was not found, silently skip it
values = parser[section]
else: # same section
values = defaults
if (raw := values.get(option)) is not None:
yield key, raw
@staticmethod
def environment() -> dict[str, str]:
"""
extract environment variables
Returns:
dict[str, str]: environment variables and some custom variables
"""
return os.environ | {
"prefix": sys.prefix,
}
def before_get(self, parser: MutableMapping[str, Mapping[str, str]], section: Any, option: Any, value: str,
defaults: Mapping[str, str]) -> str:
"""
interpolate option value
Notes:
This method is using :class:`string.Template` class in order to render both cross-references and
environment variables, because it seems that it is the most legit way to handle it. In addition,
we are using shell-like variables in some cases (see :attr:`alpm.mirror` option), thus we would like
to keep them alive.
First this method resolves substitution from the configuration and then renders environment variables
Args:
parser(MutableMapping[str, Mapping[str, str]]): option parser
section(Any): section name
option(Any): option name
value(str): source (not-converted) value
defaults(Mapping[str, str]): default values
Returns:
str: substituted value
"""
# because any substitution effectively replace escaped $ ($$) in result, we have to escape it manually
escaped = value.replace("$$", self.DATA_LINK_ESCAPE)
# resolve internal references
variables = dict(self._extract_variables(parser, value, defaults))
internal = ShellTemplate(escaped).safe_substitute(variables)
# resolve enriched environment variables by using default Template class
environment = Template(internal).safe_substitute(self.environment())
# replace escaped values back
return environment.replace(self.DATA_LINK_ESCAPE, "$")
@@ -1,155 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import fnmatch
import re
from collections.abc import Iterator, Mapping
from string import Template
class ShellTemplate(Template):
"""
extension to the default :class:`Template` class, which also adds additional tokens to braced regex and enables
bash expansion
"""
braceidpattern = r"(?a:[_a-z0-9][^}]*)"
_REMOVE_BACK = re.compile(r"^(?P<key>\w+)%(?P<pattern>.+)$")
_REMOVE_FRONT = re.compile(r"^(?P<key>\w+)#(?P<pattern>.+)$")
_REPLACE = re.compile(r"^(?P<key>\w+)/(?P<pattern>.+)/(?P<replacement>.+)$")
@staticmethod
def _remove_back(source: str, pattern: str, *, greedy: bool) -> str:
"""
resolve "${var%(%)pattern}" constructions
Args:
source(str): source string to match the pattern inside
pattern(str): shell expression to match
greedy(bool): match as much as possible or not
Returns:
str: result after removal ``pattern`` from the end of the string
"""
regex = fnmatch.translate(pattern)
compiled = re.compile(regex)
result = source
start_pos = 0
while m := compiled.search(source, start_pos):
result = source[:m.start()]
start_pos += m.start() + 1
if greedy:
break
return result
@staticmethod
def _remove_front(source: str, pattern: str, *, greedy: bool) -> str:
"""
resolve "${var#(#)pattern}" constructions
Args:
source(str): source string to match the pattern inside
pattern(str): shell expression to match
greedy(bool): match as much as possible or not
Returns:
str: result after removal ``pattern`` from the start of the string
"""
regex = fnmatch.translate(pattern)[:-2] # remove \Z at the end of the regex
if not greedy:
regex = regex.replace("*", "*?")
compiled = re.compile(regex)
m = compiled.match(source)
if m is None:
return source
return source[m.end():]
@staticmethod
def _replace(source: str, pattern: str, replacement: str, *, greedy: bool) -> str:
"""
resolve "${var/(/)pattern/replacement}" constructions
Args:
source(str): source string to match the pattern inside
pattern(str): shell expression to match
replacement(str): new substring
greedy(bool): replace as much as possible or not
Returns:
str: result after replacing ``pattern`` by ``replacement``
"""
match pattern:
case from_back if from_back.startswith("%"):
removed = ShellTemplate._remove_back(source, from_back[1:], greedy=False)
return removed if removed == source else removed + replacement
case from_front if from_front.startswith("#"):
removed = ShellTemplate._remove_front(source, from_front[1:], greedy=False)
return removed if removed == source else replacement + removed
case regular:
regex = fnmatch.translate(regular)[:-2] # remove \Z at the end of the regex
compiled = re.compile(regex)
return compiled.sub(replacement, source, count=not greedy)
def shell_substitute(self, mapping: Mapping[str, str], /, **kwargs: str) -> str:
"""
this method behaves the same as :func:`safe_substitute`, however also expands bash string operations
Args:
mapping(Mapping[str, str]): key-value dictionary of variables
**kwargs(str): key-value dictionary of variables passed as kwargs
Returns:
str: string with replaced values
"""
substitutions = (
(self._REMOVE_BACK, self._remove_back, "%"),
(self._REMOVE_FRONT, self._remove_front, "#"),
(self._REPLACE, self._replace, "/"),
)
def generator(variables: dict[str, str]) -> Iterator[tuple[str, str]]:
for identifier in self.get_identifiers():
for regex, function, greediness in substitutions:
if m := regex.match(identifier):
source = variables.get(m.group("key"))
if source is None:
continue
# replace pattern with non-greedy
pattern = m.group("pattern").removeprefix(greediness)
greedy = m.group("pattern").startswith(greediness)
# gather all additional args
args = {key: value for key, value in m.groupdict().items() if key not in ("key", "pattern")}
yield identifier, function(source, pattern, **args, greedy=greedy)
break
kwargs.update(mapping)
kwargs.update(dict(generator(kwargs)))
return self.safe_substitute(kwargs)
@@ -1,32 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
__all__ = ["steps"]
steps = [
"""
create table package_dependencies (
package_base text not null,
repository text not null,
dependencies json not null,
unique (package_base, repository)
)
""",
]
@@ -1,38 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
__all__ = ["steps"]
steps = [
"""
create table auditlog (
created integer not null,
repository text not null,
event text not null,
object_id text not null,
message text,
data json
)
""",
"""
create index auditlog_created_repository_event_object_id
on auditlog (created, repository, event, object_id)
""",
]
@@ -1,30 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
__all__ = ["steps"]
steps = [
"""
alter table logs add column process_id text not null default ''
""",
"""
alter table logs rename column record to message
""",
]
@@ -1,77 +0,0 @@
#
# Copyright (c) 2021-2025 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from dataclasses import replace
from sqlite3 import Connection
from ahriman.core.configuration import Configuration
from ahriman.core.repository import Explorer
from ahriman.core.sign.gpg import GPG
from ahriman.core.utils import atomic_move, package_like, symlink_relative
from ahriman.models.package import Package
from ahriman.models.repository_paths import RepositoryPaths
__all__ = ["migrate_data"]
def migrate_data(connection: Connection, configuration: Configuration) -> None:
"""
perform data migration
Args:
connection(Connection): database connection
configuration(Configuration): configuration instance
"""
del connection
for repository_id in Explorer.repositories_extract(configuration):
paths = replace(configuration.repository_paths, repository_id=repository_id)
# create archive directory if required
if not paths.archive.is_dir():
with paths.preserve_owner():
paths.archive.mkdir(mode=0o755, parents=True)
move_packages(paths)
def move_packages(repository_paths: RepositoryPaths) -> None:
"""
move packages from repository to archive and create symbolic links
Args:
repository_paths(RepositoryPaths): repository paths instance
"""
for archive in filter(package_like, repository_paths.repository.iterdir()):
if not archive.is_file(follow_symlinks=False):
continue # skip symbolic links if any
package = Package.from_archive(archive)
artifacts = [archive]
# check if there are signatures for this package and append it here too
if (signature := GPG.signature(archive)).exists():
artifacts.append(signature)
for source in artifacts:
target = repository_paths.ensure_exists(repository_paths.archive_for(package.base)) / source.name
# move package to the archive directory
atomic_move(source, target)
# create symlink to the archive
symlink_relative(source, target)
@@ -1,25 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
__all__ = ["steps"]
steps = [
"""alter table package_changes add column pkgbuild text""",
]
@@ -1,25 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
__all__ = ["steps"]
steps = [
"""alter table package_statuses add column is_held integer not null default 0""",
]
@@ -1,116 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from sqlite3 import Connection
from ahriman.core.database.operations.operations import Operations
from ahriman.models.dependencies import Dependencies
from ahriman.models.repository_id import RepositoryId
class DependenciesOperations(Operations):
"""
operations for dependencies table
"""
def dependencies_get(self, package_base: str | None = None,
repository_id: RepositoryId | None = None) -> dict[str, Dependencies]:
"""
get dependencies for the specific package base if available
Args:
package_base(str | None): package base to search
repository_id(RepositoryId, optional): repository unique identifier override (Default value = None)
Returns:
Dependencies: dependencies for the package base if available
"""
repository_id = repository_id or self._repository_id
def run(connection: Connection) -> dict[str, Dependencies]:
return {
row["package_base"]: Dependencies(row["dependencies"])
for row in connection.execute(
"""
select package_base, dependencies from package_dependencies
where (:package_base is null or package_base = :package_base)
and repository = :repository
""",
{
"package_base": package_base,
"repository": repository_id.id,
}
)
}
return self.with_connection(run)
def dependencies_insert(self, package_base: str, dependencies: Dependencies,
repository_id: RepositoryId | None = None) -> None:
"""
insert package dependencies
Args:
package_base(str): package base
dependencies(Dependencies): package dependencies
repository_id(RepositoryId, optional): repository unique identifier override (Default value = None)
"""
repository_id = repository_id or self._repository_id
def run(connection: Connection) -> None:
connection.execute(
"""
insert into package_dependencies
(package_base, repository, dependencies)
values
(:package_base, :repository, :dependencies)
on conflict (package_base, repository) do update set
dependencies = :dependencies
""",
{
"package_base": package_base,
"repository": repository_id.id,
"dependencies": dependencies.paths,
})
return self.with_connection(run, commit=True)
def dependencies_remove(self, package_base: str | None, repository_id: RepositoryId | None = None) -> None:
"""
remove packages dependencies
Args:
package_base(str | None): optional filter by package base
repository_id(RepositoryId, optional): repository unique identifier override (Default value = None)
"""
repository_id = repository_id or self._repository_id
def run(connection: Connection) -> None:
connection.execute(
"""
delete from package_dependencies
where (:package_base is null or package_base = :package_base)
and repository = :repository
""",
{
"package_base": package_base,
"repository": repository_id.id,
})
return self.with_connection(run, commit=True)
@@ -1,108 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from sqlite3 import Connection
from ahriman.core.database.operations.operations import Operations
from ahriman.models.event import Event, EventType
from ahriman.models.repository_id import RepositoryId
class EventOperations(Operations):
"""
operations for audit log table
"""
def event_get(self, event: str | EventType | None = None, object_id: str | None = None,
from_date: int | float | None = None, to_date: int | float | None = None,
limit: int = -1, offset: int = 0, repository_id: RepositoryId | None = None) -> list[Event]:
"""
get list of events with filters applied
Args:
event(str | EventType | None, optional): filter by event type (Default value = None)
object_id(str | None, optional): filter by event object (Default value = None)
from_date(int | float | None, optional): minimal creation date, inclusive (Default value = None)
to_date(int | float | None, optional): maximal creation date, exclusive (Default value = None)
limit(int, optional): limit records to the specified count, -1 means unlimited (Default value = -1)
offset(int, optional): records offset (Default value = 0)
repository_id(RepositoryId, optional): repository unique identifier override (Default value = None)
Returns:
list[Event]: list of audit log events
"""
repository_id = repository_id or self._repository_id
def run(connection: Connection) -> list[Event]:
return [
Event.from_json(row)
for row in connection.execute(
"""
select created, event, object_id, message, data from (
select * from auditlog
where (:event is null or event = :event)
and (:object_id is null or object_id = :object_id)
and (:from_date is null or created >= :from_date)
and (:to_date is null or created < :to_date)
and repository = :repository
order by created desc limit :limit offset :offset
) order by created asc
""",
{
"event": event,
"object_id": object_id,
"repository": repository_id.id,
"from_date": from_date,
"to_date": to_date,
"limit": limit,
"offset": offset,
}
)
]
return self.with_connection(run)
def event_insert(self, event: Event, repository_id: RepositoryId | None = None) -> None:
"""
insert audit log event
Args:
event(Event): event to insert
repository_id(RepositoryId, optional): repository unique identifier override (Default value = None)
"""
repository_id = repository_id or self._repository_id
def run(connection: Connection) -> None:
connection.execute(
"""
insert into auditlog
(created, repository, event, object_id, message, data)
values
(:created, :repository, :event, :object_id, :message, :data)
""",
{
"created": event.created,
"repository": repository_id.id,
"event": event.event,
"object_id": event.object_id,
"message": event.message,
"data": event.data,
})
return self.with_connection(run, commit=True)
@@ -1,183 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from sqlite3 import Connection
from ahriman.core.database.operations.operations import Operations
from ahriman.models.log_record import LogRecord
from ahriman.models.repository_id import RepositoryId
class LogsOperations(Operations):
"""
logs operations
"""
def logs_get(self, package_base: str, version: str | None = None, process_id: str | None = None,
limit: int = -1, offset: int = 0, repository_id: RepositoryId | None = None) -> list[LogRecord]:
"""
extract logs for specified package base
Args:
package_base(str): package base to extract logs
version(str | None, optional): package version to filter (Default value = None)
process_id(str | None, optional): process identifier to filter (Default value = None)
limit(int, optional): limit records to the specified count, -1 means unlimited (Default value = -1)
offset(int, optional): records offset (Default value = 0)
repository_id(RepositoryId, optional): repository unique identifier override (Default value = None)
Return:
list[LogRecord]: sorted package log records
"""
repository_id = repository_id or self._repository_id
def run(connection: Connection) -> list[LogRecord]:
return [
LogRecord.from_json(package_base, row)
for row in connection.execute(
"""
select created, message, version, process_id from (
select * from logs
where package_base = :package_base
and repository = :repository
and (:version is null or version = :version)
and (:process_id is null or process_id = :process_id)
order by created desc limit :limit offset :offset
) order by created asc
""",
{
"package_base": package_base,
"version": version,
"process_id": process_id,
"repository": repository_id.id,
"limit": limit,
"offset": offset,
})
]
return self.with_connection(run)
def logs_insert(self, log_record: LogRecord, repository_id: RepositoryId | None = None) -> None:
"""
write new log record to database
Args:
log_record(LogRecord): log record object
repository_id(RepositoryId, optional): repository unique identifier override (Default value = None)
"""
repository_id = repository_id or self._repository_id
def run(connection: Connection) -> None:
connection.execute(
"""
insert into logs
(package_base, version, created, message, repository, process_id)
values
(:package_base, :version, :created, :message, :repository, :process_id)
""",
{
"package_base": log_record.log_record_id.package_base,
"repository": repository_id.id,
} | log_record.view()
)
return self.with_connection(run, commit=True)
def logs_remove(self, package_base: str, version: str | None, repository_id: RepositoryId | None = None) -> None:
"""
remove log records for the specified package
Args:
package_base(str): package base to remove logs
version(str | None): package version. If set it will remove only logs belonging to another version
repository_id(RepositoryId, optional): repository unique identifier override (Default value = None)
"""
repository_id = repository_id or self._repository_id
def run(connection: Connection) -> None:
connection.execute(
"""
delete from logs
where package_base = :package_base
and repository = :repository
and (:version is null or version <> :version)
""",
{
"package_base": package_base,
"version": version,
"repository": repository_id.id,
}
)
return self.with_connection(run, commit=True)
def logs_rotate(self, keep_last_records: int, repository_id: RepositoryId | None = None) -> None:
"""
rotate logs in storage. This method will remove old logs, keeping only the last N records for each package
Args:
keep_last_records(int): number of last records to keep
repository_id(RepositoryId, optional): repository unique identifier override (Default value = None)
"""
repository_id = repository_id or self._repository_id
def remove_duplicates(connection: Connection) -> None:
connection.execute(
"""
delete from logs
where repository = :repository
and (package_base, version, repository, process_id) not in (
select package_base, version, repository, process_id from logs
where (package_base, version, repository, created) in (
select package_base, version, repository, max(created) from logs
where repository = :repository
group by package_base, version, repository
)
)
""",
{
"repository": repository_id.id,
}
)
def remove_older(connection: Connection) -> None:
connection.execute(
"""
delete from logs
where (package_base, repository, process_id) in (
select package_base, repository, process_id from (
select package_base, repository, process_id, row_number() over (partition by package_base order by max(created) desc) as rn
from logs
where repository = :repository
group by package_base, repository, process_id
)
where rn > :offset
)
""",
{
"offset": keep_last_records,
"repository": repository_id.id,
}
)
def run(connection: Connection) -> None:
remove_duplicates(connection)
remove_older(connection)
return self.with_connection(run, commit=True)
@@ -1,63 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from ahriman.core.formatters.string_printer import StringPrinter
from ahriman.models.property import Property
from ahriman.models.series_statistics import SeriesStatistics
class EventStatsPrinter(StringPrinter):
"""
print event statistics
Attributes:
statistics(SeriesStatistics): statistics object
"""
def __init__(self, event_type: str, events: list[float | int]) -> None:
"""
Args:
event_type(str): event type used for this statistics
events(list[float | int]): event values to build statistics
"""
StringPrinter.__init__(self, event_type)
self.statistics = SeriesStatistics(events)
def properties(self) -> list[Property]:
"""
convert content into printable data
Returns:
list[Property]: list of content properties
"""
properties = [
Property("total", self.statistics.total),
]
# time statistics
if self.statistics:
mean = self.statistics.mean
properties.extend([
Property("min", self.statistics.min),
Property("average", f"{mean:.3f} ± {self.statistics.st_dev:.3f}"),
Property("max", self.statistics.max),
])
return properties
@@ -1,59 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from typing import ClassVar
from ahriman.core.formatters.string_printer import StringPrinter
from ahriman.models.property import Property
class PackageStatsPrinter(StringPrinter):
"""
print packages statistics
Attributes:
MAX_COUNT(int): (class attribute) maximum number of packages to print
events(dict[str, int]): map of package to its event frequency
"""
MAX_COUNT: ClassVar[int] = 10
def __init__(self, events: dict[str, int]) -> None:
"""
Args:
events(dict[str, int]): map of package to its event frequency
"""
StringPrinter.__init__(self, "The most frequent packages")
self.events = events
def properties(self) -> list[Property]:
"""
convert content into printable data
Returns:
list[Property]: list of content properties
"""
if not self.events:
return [] # no events found, discard any stats
properties = []
for object_id, count in sorted(self.events.items(), key=lambda pair: pair[1], reverse=True)[:self.MAX_COUNT]:
properties.append(Property(object_id, count))
return properties
@@ -1,62 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from ahriman.core.formatters.printer import Printer
from ahriman.models.changes import Changes
from ahriman.models.property import Property
class PkgbuildPrinter(Printer):
"""
print content of the pkgbuild stored in changes
Attributes:
changes(Changes): package changes
"""
def __init__(self, changes: Changes) -> None:
"""
Args:
changes(Changes): package changes
"""
Printer.__init__(self)
self.changes = changes
def properties(self) -> list[Property]:
"""
convert content into printable data
Returns:
list[Property]: list of content properties
"""
if self.changes.pkgbuild is None:
return []
return [Property("", self.changes.pkgbuild, is_required=True, indent=0)]
# pylint: disable=redundant-returns-doc
def title(self) -> str | None:
"""
generate entry title from content
Returns:
str | None: content title if it can be generated and ``None`` otherwise
"""
if self.changes.pkgbuild is None:
return None
return self.changes.last_commit_sha
@@ -1,53 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from ahriman.core.formatters.string_printer import StringPrinter
from ahriman.core.utils import pretty_size
from ahriman.models.property import Property
from ahriman.models.repository_id import RepositoryId
from ahriman.models.repository_stats import RepositoryStats
class RepositoryStatsPrinter(StringPrinter):
"""
print repository statistics
Attributes:
statistics(RepositoryStats): repository statistics
"""
def __init__(self, repository_id: RepositoryId, statistics: RepositoryStats) -> None:
"""
Args:
statistics(RepositoryStats): repository statistics
"""
StringPrinter.__init__(self, str(repository_id))
self.statistics = statistics
def properties(self) -> list[Property]:
"""
convert content into printable data
Returns:
list[Property]: list of content properties
"""
return [
Property("Packages", self.statistics.bases),
Property("Repository size", pretty_size(self.statistics.archive_size)),
]
@@ -1,21 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from ahriman.core.housekeeping.archive_rotation_trigger import ArchiveRotationTrigger
from ahriman.core.housekeeping.logs_rotation_trigger import LogsRotationTrigger
@@ -1,110 +0,0 @@
#
# Copyright (c) 2021-2025 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from ahriman.core import context
from ahriman.core.configuration import Configuration
from ahriman.core.repository import Repository
from ahriman.core.triggers import Trigger
from ahriman.models.package import Package
from ahriman.models.repository_id import RepositoryId
from ahriman.models.result import Result
class ArchiveRotationTrigger(Trigger):
"""
remove packages from archive
Attributes:
keep_built_packages(int): number of last packages to keep
paths(RepositoryPaths): repository paths instance
"""
CONFIGURATION_SCHEMA = {
"archive": {
"type": "dict",
"schema": {
"keep_built_packages": {
"type": "integer",
"required": True,
"coerce": "integer",
"min": 0,
},
},
},
}
def __init__(self, repository_id: RepositoryId, configuration: Configuration) -> None:
"""
Args:
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
"""
Trigger.__init__(self, repository_id, configuration)
section = next(iter(self.configuration_sections(configuration)))
self.keep_built_packages = max(configuration.getint(section, "keep_built_packages"), 0)
self.paths = configuration.repository_paths
@classmethod
def configuration_sections(cls, configuration: Configuration) -> list[str]:
"""
extract configuration sections from configuration
Args:
configuration(Configuration): configuration instance
Returns:
list[str]: read configuration sections belong to this trigger
"""
return list(cls.CONFIGURATION_SCHEMA.keys())
def archives_remove(self, package: Package, repository: Repository) -> None:
"""
remove older versions of the specified package
Args:
package(Package): package which has been updated to check for older versions
repository(Repository): repository instance
"""
# explicit guard to skip process in case if rotation is disabled
# this guard is supposed to speedup process
if self.keep_built_packages == 0:
return
to_remove = repository.package_archives(package.base)
for single in to_remove[:-self.keep_built_packages]:
self.logger.info("removing version %s of package %s", single.version, single.base)
for archive in single.packages.values():
for path in self.paths.archive_for(single.base).glob(f"{archive.filename}*"):
path.unlink()
def on_result(self, result: Result, packages: list[Package]) -> None:
"""
run trigger
Args:
result(Result): build result
packages(list[Package]): list of all available packages
"""
ctx = context.get()
repository = ctx.get(Repository)
for package in result.success:
self.archives_remove(package, repository)
@@ -1,87 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from ahriman.core import context
from ahriman.core.configuration import Configuration
from ahriman.core.status import Client
from ahriman.core.triggers import Trigger
from ahriman.models.package import Package
from ahriman.models.repository_id import RepositoryId
from ahriman.models.result import Result
class LogsRotationTrigger(Trigger):
"""
rotate logs after build processes
Attributes:
keep_last_records(int): number of last records to keep
"""
CONFIGURATION_SCHEMA = {
"logs-rotation": {
"type": "dict",
"schema": {
"keep_last_logs": {
"type": "integer",
"required": True,
"coerce": "integer",
"min": 0,
},
},
},
}
def __init__(self, repository_id: RepositoryId, configuration: Configuration) -> None:
"""
Args:
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
"""
Trigger.__init__(self, repository_id, configuration)
section = next(iter(self.configuration_sections(configuration)))
self.keep_last_records = configuration.getint( # read old-style first and then fallback to new style
"settings", "keep_last_logs",
fallback=configuration.getint(section, "keep_last_logs"))
@classmethod
def configuration_sections(cls, configuration: Configuration) -> list[str]:
"""
extract configuration sections from configuration
Args:
configuration(Configuration): configuration instance
Returns:
list[str]: read configuration sections belong to this trigger
"""
return list(cls.CONFIGURATION_SCHEMA.keys())
def on_result(self, result: Result, packages: list[Package]) -> None:
"""
run trigger
Args:
result(Result): build result
packages(list[Package]): list of all available packages
"""
ctx = context.get()
reporter = ctx.get(Client)
reporter.logs_rotate(self.keep_last_records)
@@ -1,108 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import contextvars
import logging
from typing import Any, ClassVar, TypeVar, cast
T = TypeVar("T")
class LogContext:
"""
logging context manager which provides context variables injection into log records
"""
_context: ClassVar[dict[str, contextvars.ContextVar[Any]]] = {}
@classmethod
def get(cls, name: str) -> T | None:
"""
get context variable if available
Args:
name(str): name of the context variable
Returns:
T | None: context variable if available and ``None`` otherwise
"""
if (variable := cls._context.get(name)) is not None:
return cast(T | None, variable.get())
return None
@classmethod
def log_record_factory(cls, *args: Any, **kwargs: Any) -> logging.LogRecord:
"""
log record factory which injects all registered context variables into log records
Args:
*args(Any): positional arguments for the log factory
**kwargs(Any): keyword arguments for the log factory
Returns:
logging.LogRecord: log record with context variables set as attributes
"""
record = logging.LogRecord(*args, **kwargs)
for name, variable in cls._context.items():
if (value := variable.get()) is not None:
setattr(record, name, value)
return record
@classmethod
def register(cls, name: str) -> contextvars.ContextVar[T]:
"""
(re)create context variable for log records
Args:
name(str): name of the context variable
Returns:
contextvars.ContextVar[T]: created context variable
"""
variable = cls._context[name] = contextvars.ContextVar(name, default=None)
return variable
@classmethod
def reset(cls, name: str, token: contextvars.Token[T]) -> None:
"""
reset context variable to its previous value
Args:
name(str): attribute name to reset on log records
token(contextvars.Token[T]): previously registered token
"""
cls._context[name].reset(token)
@classmethod
def set(cls, name: str, value: T) -> contextvars.Token[T]:
"""
set context variable for log records. This value will be automatically emitted with each log record
Args:
name(str): attribute name to set on log records
value(T): current value of the context variable
Returns:
contextvars.Token[T]: token created with this value
"""
return cls._context[name].set(value)
@@ -1,76 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import inspect
from collections.abc import Iterator
from importlib import import_module
from pathlib import Path
from pkgutil import ModuleInfo, walk_packages
from types import ModuleType
from typing import Any, TypeGuard, TypeVar
__all__ = ["implementations"]
T = TypeVar("T")
def _modules(module_root: Path, prefix: str) -> Iterator[ModuleInfo]:
"""
extract available modules from package
Args:
module_root(Path): module root path
prefix(str): modules package prefix
Yields:
ModuleInfo: module information each available module
"""
prefix = f"{prefix}." if prefix else ""
for module_info in walk_packages([str(module_root)], prefix):
if not module_info.ispkg:
yield module_info
def implementations(root_module: ModuleType, base_class: type[T]) -> Iterator[type[T]]:
"""
extract implementations of the ``base_class`` from the module
Args:
root_module(ModuleType): root module
base_class(type[T]): base class type
Yields:
type[T]: found implementations
"""
def is_base_class(clazz: Any) -> TypeGuard[type[T]]:
return inspect.isclass(clazz) \
and issubclass(clazz, base_class) and clazz != base_class \
and clazz.__module__ == module.__name__
prefix = root_module.__name__
for module_root in root_module.__path__:
for module_info in _modules(Path(module_root), prefix):
module = import_module(module_info.name)
for _, attribute in inspect.getmembers(module, is_base_class):
yield attribute
-129
View File
@@ -1,129 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import datetime
from collections.abc import Callable
from email.utils import format_datetime, parsedate_to_datetime
from typing import Any
from ahriman.core import context
from ahriman.core.configuration import Configuration
from ahriman.core.report.jinja_template import JinjaTemplate
from ahriman.core.report.report import Report
from ahriman.core.status import Client
from ahriman.core.types import Comparable
from ahriman.models.event import EventType
from ahriman.models.package import Package
from ahriman.models.repository_id import RepositoryId
from ahriman.models.result import Result
class RSS(Report, JinjaTemplate):
"""
RSS report generator
Attributes:
max_entries(int): the maximal amount of entries in RSS
report_path(Path): output path to RSS report
template(str): name of the template
"""
def __init__(self, repository_id: RepositoryId, configuration: Configuration, section: str) -> None:
"""
Args:
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
section(str): settings section name
"""
Report.__init__(self, repository_id, configuration)
JinjaTemplate.__init__(self, repository_id, configuration, section)
self.max_entries = configuration.getint(section, "max_entries", fallback=-1)
self.report_path = configuration.getpath(section, "path")
self.template = configuration.get(section, "template")
@staticmethod
def format_datetime(timestamp: datetime.datetime | float | int | None) -> str:
"""
convert datetime object to string
Args:
timestamp(datetime.datetime | float | int | None): datetime to convert
Returns:
str: datetime as string representation
"""
if timestamp is None:
return ""
if isinstance(timestamp, (int, float)):
timestamp = datetime.datetime.fromtimestamp(timestamp, datetime.UTC)
return format_datetime(timestamp)
@staticmethod
def sort_content(content: list[dict[str, Any]]) -> list[dict[str, Any]]:
"""
sort content before rendering
Args:
content(list[dict[str, str]]): content of the template
Returns:
list[dict[str, str]]: sorted content according to comparator defined
"""
comparator: Callable[[dict[str, str]], Comparable] = \
lambda item: parsedate_to_datetime(item["build_date"])
return sorted(content, key=comparator, reverse=True)
def content(self, packages: list[Package]) -> Result:
"""
extract result to be written to template
Args:
packages(list[Package]): list of packages to generate report
Returns:
Result: result descriptor
"""
ctx = context.get()
reporter = ctx.get(Client)
events = reporter.event_get(EventType.PackageUpdated, None, limit=self.max_entries)
known_packages = {package.base: package for package in packages}
result = Result()
for event in events:
package = known_packages.get(event.object_id)
if package is None:
continue # package not found
result.add_updated(package)
return result
def generate(self, packages: list[Package], result: Result) -> None:
"""
generate report for the specified packages
Args:
packages(list[Package]): list of packages to generate report
result(Result): build result
"""
result = self.content(packages)
rss = self.make_html(result, self.template)
self.report_path.write_text(rss, encoding="utf8")
@@ -1,84 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import contextlib
from collections.abc import Iterator
from ahriman.core.status import Client
from ahriman.models.event import Event, EventType
from ahriman.models.metrics_timer import MetricsTimer
class EventLogger:
"""
wrapper for logging events
Attributes:
reporter(Client): build status reporter instance
"""
reporter: Client
def event(self, package_base: str, event: EventType, message: str | None = None) -> None:
"""
log single event. For timed events use context manager :func:`in_event()` instead
Args:
package_base(str): package base name
event(EventType): event type to be logged on success action
message(str | None, optional): optional message describing the action (Default value = None)
Examples:
This method must be used as simple wrapper for :class:`ahriman.core.status.Client` methods, e.g.::
>>> do_something()
>>> self.event(package_base, EventType.PackageUpdated)
"""
self.reporter.event_add(Event(event, package_base, message))
@contextlib.contextmanager
def in_event(self, package_base: str, event: EventType, message: str | None = None,
failure: EventType | None = None) -> Iterator[None]:
"""
perform action in package context and log event with time elapsed
Args:
package_base(str): package base name
event(EventType): event type to be logged on success action
message(str | None, optional): optional message describing the action (Default value = None)
failure(EventType | None, optional): event type to be logged on exception (Default value = None)
Examples:
This method must be used to perform action in context with time measurement::
>>> with self.in_event(package_base, EventType.PackageUpdated):
>>> do_something()
Additional parameter ``failure`` can be set in order to emit an event on exception occurred. If none set
(default), then no event will be recorded on exception
"""
with MetricsTimer() as timer:
try:
yield
self.reporter.event_add(Event(event, package_base, message, took=timer.elapsed))
except Exception:
if failure is not None:
self.reporter.event_add(Event(failure, package_base, took=timer.elapsed))
raise
@@ -1,70 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from collections.abc import Iterable
from ahriman.core.configuration import Configuration
from ahriman.models.repository_id import RepositoryId
from ahriman.models.repository_paths import RepositoryPaths
class Explorer:
"""
helper to read filesystem and find created repositories
"""
@staticmethod
def repositories_extract(configuration: Configuration, repository: str | None = None,
architecture: str | None = None) -> list[RepositoryId]:
"""
get known architectures
Args:
configuration(Configuration): configuration instance
repository(str | None, optional): predefined repository name if available (Default value = None)
architecture(str | None, optional): predefined repository architecture if available (Default value = None)
Returns:
list[RepositoryId]: list of repository names and architectures for which tree is created
"""
# pylint, wtf???
root = configuration.getpath("repository", "root") # pylint: disable=assignment-from-no-return
# extract repository names first
if repository is not None:
repositories: Iterable[str] = [repository]
elif from_filesystem := RepositoryPaths.known_repositories(root):
repositories = from_filesystem
else: # try to read configuration now
repositories = [configuration.get("repository", "name")]
# extract architecture names
if architecture is not None:
parsed = set(
RepositoryId(architecture, repository)
for repository in repositories
)
else: # try to read from file system
parsed = set(
RepositoryId(architecture, repository)
for repository in repositories
for architecture in RepositoryPaths.known_architectures(root, repository)
)
return sorted(parsed)
@@ -1,239 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import copy
from collections.abc import Callable, Iterable
from functools import cmp_to_key
from pathlib import Path
from tempfile import TemporaryDirectory
from ahriman.core.alpm.pacman import Pacman
from ahriman.core.build_tools.sources import Sources
from ahriman.core.configuration import Configuration
from ahriman.core.log import LazyLogging
from ahriman.core.status import Client
from ahriman.core.utils import list_flatmap, package_like
from ahriman.models.changes import Changes
from ahriman.models.package import Package
from ahriman.models.repository_id import RepositoryId
from ahriman.models.repository_paths import RepositoryPaths
class PackageInfo(LazyLogging):
"""
handler for the package information
Attributes:
configuration(Configuration): configuration instance
pacman(Pacman): alpm wrapper instance
paths(RepositoryPaths): repository paths instance
reporter(Client): build status reporter instance
repository_id(RepositoryId): repository unique identifier
"""
configuration: Configuration
pacman: Pacman
paths: RepositoryPaths
reporter: Client
repository_id: RepositoryId
def full_depends(self, package: Package, packages: Iterable[Package]) -> list[str]:
"""
generate full dependencies list including transitive dependencies
Args:
package(Package): package to check dependencies for
packages(Iterable[Package]): repository package list
Returns:
list[str]: all dependencies of the package
"""
dependencies = {}
# load own package dependencies
for package_base in packages:
for name, repo_package in package_base.packages.items():
dependencies[name] = repo_package.depends
for provides in repo_package.provides:
dependencies[provides] = repo_package.depends
# load repository dependencies
for database in self.pacman.handle.get_syncdbs():
for pacman_package in database.pkgcache:
dependencies[pacman_package.name] = pacman_package.depends
for provides in pacman_package.provides:
dependencies[provides] = pacman_package.depends
result = set(package.depends)
current_depends: set[str] = set()
while result != current_depends:
current_depends = copy.deepcopy(result)
for package_name in current_depends:
result.update(dependencies.get(package_name, []))
return sorted(result)
def load_archives(self, packages: Iterable[Path], *, latest_only: bool = True) -> list[Package]:
"""
load packages from list of archives
Args:
packages(Iterable[Path]): paths to package archives
latest_only(bool, optional): filter packages with the same base, keeping only fresh packages installed
(Default value = True)
Returns:
list[Package]: list of read packages
"""
sources = {package.base: package.remote for package, _, in self.reporter.package_get(None)}
result: dict[str, dict[str, Package]] = {}
# we are iterating over bases, not single packages
for full_path in packages:
try:
local = Package.from_archive(full_path)
if (source := sources.get(local.base)) is not None: # update source with remote
local.remote = source
loaded_versions = result.setdefault(local.base, {})
current = loaded_versions.setdefault(local.version, local)
current.packages.update(local.packages)
except Exception:
self.logger.exception("could not load package from %s", full_path)
if latest_only:
comparator: Callable[[Package, Package], int] = lambda left, right: left.vercmp(right.version)
for package_base, versions in result.items():
newest = max(versions.values(), key=cmp_to_key(comparator))
result[package_base] = {newest.version: newest}
return [
package
for versions in result.values()
for package in versions.values()
]
def package_archives(self, package_base: str) -> list[Package]:
"""
load list of packages known for this package base. This method unlike
:func:`ahriman.core.repository.package_info.PackageInfo.load_archives` scans archive directory and loads all
versions available for the ``package_base``
Args:
package_base(str): package base
Returns:
list[Package]: list of packages belonging to this base, sorted by version by ascension
"""
archive = self.paths.archive_for(package_base)
if not archive.is_dir():
return []
packages = self.load_archives(filter(package_like, archive.iterdir()), latest_only=False)
comparator: Callable[[Package, Package], int] = lambda left, right: left.vercmp(right.version)
return sorted(
(package for package in packages if package.supports_architecture(self.repository_id.architecture)),
key=cmp_to_key(comparator),
)
def package_archives_lookup(self, package: Package) -> list[Path]:
"""
check if there is a rebuilt package already
Args:
package(Package): package to check
Returns:
list[Path]: list of built packages and signatures if available, empty list otherwise
"""
archive = self.paths.archive_for(package.base)
for built in self.package_archives(package.base):
if built.version != package.version:
continue
return list_flatmap(built.packages.values(), lambda single: archive.glob(f"{single.filename}*"))
return []
def package_changes(self, package: Package, last_commit_sha: str) -> Changes | None:
"""
extract package change for the package since last commit if available
Args:
package(Package): package properties
last_commit_sha(str): last known commit hash
Returns:
Changes | None: changes if available
"""
with TemporaryDirectory(ignore_cleanup_errors=True) as dir_name:
dir_path = Path(dir_name)
patches = self.reporter.package_patches_get(package.base, None)
current_commit_sha = Sources.load(dir_path, package, patches, self.paths)
if current_commit_sha != last_commit_sha:
return Sources.changes(dir_path, last_commit_sha)
return None
def packages(self, filter_packages: Iterable[str] | None = None) -> list[Package]:
"""
generate list of repository packages
Args:
filter_packages(Iterable[str] | None, optional): filter packages list by specified only
Returns:
list[Package]: list of packages properties
"""
packages = self.load_archives(filter(package_like, self.paths.repository.iterdir()))
if filter_packages:
packages = [package for package in packages if package.base in filter_packages]
return packages
def packages_built(self) -> list[Path]:
"""
get list of files in built packages directory
Returns:
list[Path]: list of filenames from the directory
"""
return list(filter(package_like, self.paths.packages.iterdir()))
def packages_depend_on(self, packages: list[Package], depends_on: Iterable[str] | None) -> list[Package]:
"""
extract list of packages which depends on specified package
Args:
packages(list[Package]): list of packages to be filtered
depends_on(Iterable[str] | None): dependencies of the packages
Returns:
list[Package]: list of repository packages which depend on specified packages
"""
if depends_on is None:
return packages # no list provided extract everything by default
depends_on = set(depends_on)
return [
package
for package in packages
if depends_on.intersection(self.full_depends(package, packages))
]
@@ -1,19 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
@@ -1,20 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from ahriman.core.status.client import Client
@@ -1,410 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# pylint: disable=too-many-public-methods
from __future__ import annotations
from ahriman.core.configuration import Configuration
from ahriman.core.database import SQLite
from ahriman.models.build_status import BuildStatus, BuildStatusEnum
from ahriman.models.changes import Changes
from ahriman.models.dependencies import Dependencies
from ahriman.models.event import Event, EventType
from ahriman.models.internal_status import InternalStatus
from ahriman.models.log_record import LogRecord
from ahriman.models.package import Package
from ahriman.models.pkgbuild_patch import PkgbuildPatch
from ahriman.models.repository_id import RepositoryId
from ahriman.models.repository_stats import RepositoryStats
class Client:
"""
base build status reporter client
"""
@staticmethod
def load(repository_id: RepositoryId, configuration: Configuration, database: SQLite | None = None, *,
report: bool = True) -> Client:
"""
load client from settings
Args:
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
database(SQLite | None, optional): database instance (Default value = None)
report(bool, optional): force enable or disable reporting (Default value = True)
Returns:
Client: client according to current settings
"""
def make_local_client() -> Client:
if database is None:
return Client()
from ahriman.core.status.local_client import LocalClient
return LocalClient(repository_id, database)
if not report:
return make_local_client()
if not configuration.getboolean("status", "enabled", fallback=True): # global switch
return make_local_client()
# new-style section
address = configuration.get("status", "address", fallback=None)
# old-style section
legacy_address = configuration.get("web", "address", fallback=None)
host = configuration.get("web", "host", fallback=None)
port = configuration.getint("web", "port", fallback=None)
socket = configuration.get("web", "unix_socket", fallback=None)
# basically we just check if there is something we can use for interaction with remote server
if address or legacy_address or (host and port) or socket:
from ahriman.core.status.web_client import WebClient
return WebClient(repository_id, configuration)
return make_local_client()
def configuration_reload(self) -> None:
"""
reload configuration
"""
def event_add(self, event: Event) -> None:
"""
create new event
Args:
event(Event): audit log event
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def event_get(self, event: str | EventType | None, object_id: str | None,
from_date: int | float | None = None, to_date: int | float | None = None,
limit: int = -1, offset: int = 0) -> list[Event]:
"""
retrieve list of events
Args:
event(str | EventType | None): filter by event type
object_id(str | None): filter by event object
from_date(int | float | None, optional): minimal creation date, inclusive (Default value = None)
to_date(int | float | None, optional): maximal creation date, exclusive (Default value = None)
limit(int, optional): limit records to the specified count, -1 means unlimited (Default value = -1)
offset(int, optional): records offset (Default value = 0)
Returns:
list[Event]: list of audit log events
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def logs_rotate(self, keep_last_records: int) -> None:
"""
remove older logs from storage
Args:
keep_last_records(int): number of last records to keep
"""
def package_changes_get(self, package_base: str) -> Changes:
"""
get package changes
Args:
package_base(str): package base to retrieve
Returns:
Changes: package changes if available and empty object otherwise
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def package_changes_update(self, package_base: str, changes: Changes) -> None:
"""
update package changes
Args:
package_base(str): package base to update
changes(Changes): changes descriptor
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def package_dependencies_get(self, package_base: str) -> Dependencies:
"""
get package dependencies
Args:
package_base(str): package base to retrieve
Returns:
list[Dependencies]: package implicit dependencies if available
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def package_dependencies_update(self, package_base: str, dependencies: Dependencies) -> None:
"""
update package dependencies
Args:
package_base(str): package base to update
dependencies(Dependencies): dependencies descriptor
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def package_get(self, package_base: str | None) -> list[tuple[Package, BuildStatus]]:
"""
get package status
Args:
package_base(str | None): package base to get
Returns:
list[tuple[Package, BuildStatus]]: list of current package description and status if it has been found
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def package_hold_update(self, package_base: str, *, enabled: bool) -> None:
"""
update package hold status
Args:
package_base(str): package base name
enabled(bool): new hold status
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def package_logs_add(self, log_record: LogRecord) -> None:
"""
post log record
Args:
log_record(LogRecord): log record
"""
# this method does not raise NotImplementedError because it is actively used as dummy client for http log
def package_logs_get(self, package_base: str, version: str | None = None, process_id: str | None = None,
limit: int = -1, offset: int = 0) -> list[LogRecord]:
"""
get package logs
Args:
package_base(str): package base
version(str | None, optional): package version to search (Default value = None)
process_id(str | None, optional): process identifier to search (Default value = None)
limit(int, optional): limit records to the specified count, -1 means unlimited (Default value = -1)
offset(int, optional): records offset (Default value = 0)
Returns:
list[LogRecord]: package logs
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def package_logs_remove(self, package_base: str, version: str | None) -> None:
"""
remove package logs
Args:
package_base(str): package base
version(str | None): package version to remove logs. If ``None`` is set, all logs will be removed
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def package_patches_get(self, package_base: str, variable: str | None) -> list[PkgbuildPatch]:
"""
get package patches
Args:
package_base(str): package base to retrieve
variable(str | None): optional filter by patch variable
Returns:
list[PkgbuildPatch]: list of patches for the specified package
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def package_patches_remove(self, package_base: str, variable: str | None) -> None:
"""
remove package patch
Args:
package_base(str): package base to update
variable(str | None): patch name. If ``None`` is set, all patches will be removed
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def package_patches_update(self, package_base: str, patch: PkgbuildPatch) -> None:
"""
create or update package patch
Args:
package_base(str): package base to update
patch(PkgbuildPatch): package patch
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def package_remove(self, package_base: str) -> None:
"""
remove packages from watcher
Args:
package_base(str): package base to remove
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def package_status_update(self, package_base: str, status: BuildStatusEnum) -> None:
"""
update package build status. Unlike :func:`package_update()` it does not update package properties
Args:
package_base(str): package base to update
status(BuildStatusEnum): current package build status
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def package_update(self, package: Package, status: BuildStatusEnum) -> None:
"""
add new package or update existing one with status
Args:
package(Package): package properties
status(BuildStatusEnum): current package build status
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def set_building(self, package_base: str) -> None:
"""
set package status to building
Args:
package_base(str): package base to update
"""
self.package_status_update(package_base, BuildStatusEnum.Building)
def set_failed(self, package_base: str) -> None:
"""
set package status to failed
Args:
package_base(str): package base to update
"""
self.package_status_update(package_base, BuildStatusEnum.Failed)
def set_pending(self, package_base: str) -> None:
"""
set package status to pending
Args:
package_base(str): package base to update
"""
self.package_status_update(package_base, BuildStatusEnum.Pending)
def set_success(self, package: Package) -> None:
"""
set package status to success
Args:
package(Package): current package properties
"""
self.package_update(package, BuildStatusEnum.Success)
def set_unknown(self, package: Package) -> None:
"""
set package status to unknown. Unlike other methods, this method also checks if package is known,
and - in case if it is - it silently skips update
Args:
package(Package): current package properties
"""
if self.package_get(package.base):
return # skip update in case if package is already known
self.package_update(package, BuildStatusEnum.Unknown)
def statistics(self) -> RepositoryStats:
"""
get repository statistics
Returns:
RepositoryStats: repository statistics object
"""
packages = [package for package, _ in self.package_get(None)]
return RepositoryStats.from_packages(packages)
def status_get(self) -> InternalStatus:
"""
get internal service status
Returns:
InternalStatus: current internal (web) service status
"""
return InternalStatus(status=BuildStatus())
def status_update(self, status: BuildStatusEnum) -> None:
"""
update ahriman status itself
Args:
status(BuildStatusEnum): current ahriman status
"""
@@ -1,136 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import uuid
from asyncio import Lock, Queue, QueueFull, QueueShutDown
from dataclasses import dataclass
from typing import Any
from ahriman.core.log import LazyLogging
from ahriman.models.event import EventType
SSEvent = tuple[str, dict[str, Any]]
@dataclass(frozen=True)
class _Subscription:
"""
internal event bus subscription record
Attributes:
topics(list[EventType] | None): event type filter, ``None`` means all
object_id(str | None): object identifier filter, ``None`` means all
queue(Queue[SSEvent]): per-subscriber event queue
"""
topics: list[EventType] | None
object_id: str | None
queue: Queue[SSEvent]
class EventBus(LazyLogging):
"""
event bus implementation
Attributes:
max_size(int): maximum size of queue
"""
def __init__(self, max_size: int) -> None:
"""
Args:
max_size(int): maximum size of queue
"""
self.max_size = max_size
self._lock = Lock()
self._subscribers: dict[str, _Subscription] = {}
async def broadcast(self, event_type: EventType, object_id: str | None, **kwargs: Any) -> None:
"""
broadcast event to all subscribers
Args:
event_type(EventType): event type
object_id(str | None): object identifier (e.g. package base)
**kwargs(Any): additional event data
"""
event: dict[str, Any] = {"object_id": object_id}
event.update(kwargs)
async with self._lock:
snapshot = list(self._subscribers.items())
for subscriber_id, subscription in snapshot:
if subscription.topics is not None and event_type not in subscription.topics:
continue
if subscription.object_id is not None and object_id != subscription.object_id:
continue
try:
subscription.queue.put_nowait((event_type, event))
except QueueFull:
self.logger.warning("discard message to slow subscriber %s", subscriber_id)
except QueueShutDown:
pass
async def shutdown(self) -> None:
"""
gracefully shutdown all subscribers
"""
async with self._lock:
for subscription in self._subscribers.values():
subscription.queue.shutdown()
self._subscribers.clear()
async def subscribe(self, topics: list[EventType] | None = None,
object_id: str | None = None) -> tuple[str, Queue[SSEvent]]:
"""
register new subscriber
Args:
topics(list[EventType] | None, optional): list of event types to filter by. If ``None`` is set,
all events will be delivered (Default value = None)
object_id(str | None, optional): object identifier to filter by. If ``None`` is set,
events for all objects will be delivered (Default value = None)
Returns:
tuple[str, Queue[SSEvent]]: subscriber identifier and associated queue
"""
subscriber_id = str(uuid.uuid4())
queue: Queue[SSEvent] = Queue(self.max_size)
async with self._lock:
self._subscribers[subscriber_id] = _Subscription(topics=topics, object_id=object_id, queue=queue)
return subscriber_id, queue
async def unsubscribe(self, subscriber_id: str) -> None:
"""
unsubscribe from events
Args:
subscriber_id(str): subscriber unique identifier
"""
async with self._lock:
subscription = self._subscribers.pop(subscriber_id, None)
if subscription is not None:
subscription.queue.shutdown()
@@ -1,261 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from ahriman.core.database import SQLite
from ahriman.core.status import Client
from ahriman.models.build_status import BuildStatus, BuildStatusEnum
from ahriman.models.changes import Changes
from ahriman.models.dependencies import Dependencies
from ahriman.models.event import Event, EventType
from ahriman.models.log_record import LogRecord
from ahriman.models.package import Package
from ahriman.models.pkgbuild_patch import PkgbuildPatch
from ahriman.models.repository_id import RepositoryId
class LocalClient(Client):
"""
local database handler
Attributes:
database(SQLite): database instance
repository_id(RepositoryId): repository unique identifier
"""
def __init__(self, repository_id: RepositoryId, database: SQLite) -> None:
"""
Args:
repository_id(RepositoryId): repository unique identifier
database(SQLite): database instance
"""
self.database = database
self.repository_id = repository_id
def event_add(self, event: Event) -> None:
"""
create new event
Args:
event(Event): audit log event
"""
self.database.event_insert(event, self.repository_id)
def event_get(self, event: str | EventType | None, object_id: str | None,
from_date: int | float | None = None, to_date: int | float | None = None,
limit: int = -1, offset: int = 0) -> list[Event]:
"""
retrieve list of events
Args:
event(str | EventType | None): filter by event type
object_id(str | None): filter by event object
from_date(int | float | None, optional): minimal creation date, inclusive (Default value = None)
to_date(int | float | None, optional): maximal creation date, exclusive (Default value = None)
limit(int, optional): limit records to the specified count, -1 means unlimited (Default value = -1)
offset(int, optional): records offset (Default value = 0)
Returns:
list[Event]: list of audit log events
"""
return self.database.event_get(event, object_id, from_date, to_date, limit, offset, self.repository_id)
def logs_rotate(self, keep_last_records: int) -> None:
"""
remove older logs from storage
Args:
keep_last_records(int): number of last records to keep
"""
self.database.logs_rotate(keep_last_records, self.repository_id)
def package_changes_get(self, package_base: str) -> Changes:
"""
get package changes
Args:
package_base(str): package base to retrieve
Returns:
Changes: package changes if available and empty object otherwise
"""
return self.database.changes_get(package_base, self.repository_id)
def package_changes_update(self, package_base: str, changes: Changes) -> None:
"""
update package changes
Args:
package_base(str): package base to update
changes(Changes): changes descriptor
"""
self.database.changes_insert(package_base, changes, self.repository_id)
def package_dependencies_get(self, package_base: str) -> Dependencies:
"""
get package dependencies
Args:
package_base(str): package base to retrieve
Returns:
list[Dependencies]: package implicit dependencies if available
"""
return self.database.dependencies_get(package_base, self.repository_id).get(package_base, Dependencies())
def package_dependencies_update(self, package_base: str, dependencies: Dependencies) -> None:
"""
update package dependencies
Args:
package_base(str): package base to update
dependencies(Dependencies): dependencies descriptor
"""
self.database.dependencies_insert(package_base, dependencies, self.repository_id)
def package_get(self, package_base: str | None) -> list[tuple[Package, BuildStatus]]:
"""
get package status
Args:
package_base(str | None): package base to get
Returns:
list[tuple[Package, BuildStatus]]: list of current package description and status if it has been found
"""
packages = self.database.packages_get(self.repository_id)
if package_base is None:
return packages
return [(package, status) for package, status in packages if package.base == package_base]
def package_hold_update(self, package_base: str, *, enabled: bool) -> None:
"""
update package hold status
Args:
package_base(str): package base name
enabled(bool): new hold status
"""
self.database.package_hold_update(package_base, self.repository_id, enabled=enabled)
def package_logs_add(self, log_record: LogRecord) -> None:
"""
post log record
Args:
log_record(LogRecord): log record
"""
self.database.logs_insert(log_record, self.repository_id)
def package_logs_get(self, package_base: str, version: str | None = None, process_id: str | None = None,
limit: int = -1, offset: int = 0) -> list[LogRecord]:
"""
get package logs
Args:
package_base(str): package base
version(str | None, optional): package version to search (Default value = None)
process_id(str | None, optional): process identifier to search (Default value = None)
limit(int, optional): limit records to the specified count, -1 means unlimited (Default value = -1)
offset(int, optional): records offset (Default value = 0)
Returns:
list[LogRecord]: package logs
"""
return self.database.logs_get(package_base, version, process_id, limit, offset, self.repository_id)
def package_logs_remove(self, package_base: str, version: str | None) -> None:
"""
remove package logs
Args:
package_base(str): package base
version(str | None): package version to remove logs. If ``None`` is set, all logs will be removed
"""
self.database.logs_remove(package_base, version, self.repository_id)
def package_patches_get(self, package_base: str, variable: str | None) -> list[PkgbuildPatch]:
"""
get package patches
Args:
package_base(str): package base to retrieve
variable(str | None): optional filter by patch variable
Returns:
list[PkgbuildPatch]: list of patches for the specified package
"""
variables = [variable] if variable is not None else None
return self.database.patches_list(package_base, variables).get(package_base, [])
def package_patches_remove(self, package_base: str, variable: str | None) -> None:
"""
remove package patch
Args:
package_base(str): package base to update
variable(str | None): patch name. If ``None`` is set, all patches will be removed
"""
variables = [variable] if variable is not None else None
self.database.patches_remove(package_base, variables)
def package_patches_update(self, package_base: str, patch: PkgbuildPatch) -> None:
"""
create or update package patch
Args:
package_base(str): package base to update
patch(PkgbuildPatch): package patch
"""
self.database.patches_insert(package_base, [patch])
def package_remove(self, package_base: str) -> None:
"""
remove packages from watcher
Args:
package_base(str): package base to remove
"""
self.database.package_clear(package_base, self.repository_id)
def package_status_update(self, package_base: str, status: BuildStatusEnum) -> None:
"""
update package build status. Unlike :func:`package_update()` it does not update package properties
Args:
package_base(str): package base to update
status(BuildStatusEnum): current package build status
Raises:
NotImplementedError: not implemented method
"""
self.database.status_update(package_base, BuildStatus(status), self.repository_id)
def package_update(self, package: Package, status: BuildStatusEnum) -> None:
"""
add new package or update existing one with status
Args:
package(Package): package properties
status(BuildStatusEnum): current package build status
Raises:
NotImplementedError: not implemented method
"""
self.database.package_update(package, self.repository_id)
self.database.status_update(package.base, BuildStatus(status), self.repository_id)
@@ -1,361 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# pylint: disable=too-many-public-methods
from asyncio import Lock
from dataclasses import replace
from typing import Self
from ahriman.core.exceptions import UnknownPackageError
from ahriman.core.log import LazyLogging
from ahriman.core.repository.package_info import PackageInfo
from ahriman.core.status import Client
from ahriman.core.status.event_bus import EventBus
from ahriman.models.build_status import BuildStatus, BuildStatusEnum
from ahriman.models.changes import Changes
from ahriman.models.dependencies import Dependencies
from ahriman.models.event import Event, EventType
from ahriman.models.log_record import LogRecord
from ahriman.models.package import Package
from ahriman.models.pkgbuild_patch import PkgbuildPatch
class Watcher(LazyLogging):
"""
package status watcher
Attributes:
client(Client): reporter instance
event_bus(EventBus): event bus instance
package_info(PackageInfo): package info instance
status(BuildStatus): daemon status
"""
def __init__(self, client: Client, package_info: PackageInfo, event_bus: EventBus) -> None:
"""
Args:
client(Client): reporter instance
package_info(PackageInfo): package info instance
event_bus(EventBus): event bus instance
"""
self.client = client
self.package_info = package_info
self.event_bus = event_bus
self._lock = Lock()
self._known: dict[str, tuple[Package, BuildStatus]] = {}
self.status = BuildStatus()
async def event_add(self, event: Event) -> None:
"""
create new event
Args:
event(Event): audit log event
"""
self.client.event_add(event)
async def event_get(self, event: str | EventType | None, object_id: str | None,
from_date: int | float | None = None, to_date: int | float | None = None,
limit: int = -1, offset: int = 0) -> list[Event]:
"""
retrieve list of events
Args:
event(str | EventType | None): filter by event type
object_id(str | None): filter by event object
from_date(int | float | None, optional): minimal creation date, inclusive (Default value = None)
to_date(int | float | None, optional): maximal creation date, exclusive (Default value = None)
limit(int, optional): limit records to the specified count, -1 means unlimited (Default value = -1)
offset(int, optional): records offset (Default value = 0)
Returns:
list[Event]: list of audit log events
"""
return self.client.event_get(event, object_id, from_date, to_date, limit, offset)
async def load(self) -> None:
"""
load packages from local database
"""
async with self._lock:
self._known = {
package.base: (package, status)
for package, status in self.client.package_get(None)
}
async def logs_rotate(self, keep_last_records: int) -> None:
"""
remove older logs from storage
Args:
keep_last_records(int): number of last records to keep
"""
self.client.logs_rotate(keep_last_records)
async def package_archives(self, package_base: str) -> list[Package]:
"""
get known package archives
Args:
package_base(str): package base
Returns:
list[Package]: list of built package for this package base
"""
return self.package_info.package_archives(package_base)
async def package_changes_get(self, package_base: str) -> Changes:
"""
get package changes
Args:
package_base(str): package base to retrieve
Returns:
Changes: package changes if available and empty object otherwise
"""
return self.client.package_changes_get(package_base)
async def package_changes_update(self, package_base: str, changes: Changes) -> None:
"""
update package changes
Args:
package_base(str): package base to update
changes(Changes): changes descriptor
"""
self.client.package_changes_update(package_base, changes)
async def package_dependencies_get(self, package_base: str) -> Dependencies:
"""
get package dependencies
Args:
package_base(str): package base to retrieve
Returns:
list[Dependencies]: package implicit dependencies if available
"""
return self.client.package_dependencies_get(package_base)
async def package_dependencies_update(self, package_base: str, dependencies: Dependencies) -> None:
"""
update package dependencies
Args:
package_base(str): package base to update
dependencies(Dependencies): dependencies descriptor
"""
self.client.package_dependencies_update(package_base, dependencies)
async def package_get(self, package_base: str) -> tuple[Package, BuildStatus]:
"""
get current package base build status
Args:
package_base(str): package base
Returns:
tuple[Package, BuildStatus]: package and its status
Raises:
UnknownPackageError: if no package found
"""
try:
async with self._lock:
return self._known[package_base]
except KeyError:
raise UnknownPackageError(package_base) from None
async def package_hold_update(self, package_base: str, *, enabled: bool) -> None:
"""
update package hold status
Args:
package_base(str): package base name
enabled(bool): new hold status
"""
package, status = await self.package_get(package_base)
async with self._lock:
self._known[package_base] = (package, replace(status, is_held=enabled))
self.client.package_hold_update(package_base, enabled=enabled)
await self.event_bus.broadcast(EventType.PackageHeld, package_base, is_held=enabled)
async def package_logs_add(self, log_record: LogRecord) -> None:
"""
post log record
Args:
log_record(LogRecord): log record
"""
self.client.package_logs_add(log_record)
await self.event_bus.broadcast(EventType.BuildLog, log_record.log_record_id.package_base, **log_record.view())
async def package_logs_get(self, package_base: str, version: str | None = None, process_id: str | None = None,
limit: int = -1, offset: int = 0) -> list[LogRecord]:
"""
get package logs
Args:
package_base(str): package base
version(str | None, optional): package version to search (Default value = None)
process_id(str | None, optional): process identifier to search (Default value = None)
limit(int, optional): limit records to the specified count, -1 means unlimited (Default value = -1)
offset(int, optional): records offset (Default value = 0)
Returns:
list[LogRecord]: package logs
"""
return self.client.package_logs_get(package_base, version, process_id, limit, offset)
async def package_logs_remove(self, package_base: str, version: str | None) -> None:
"""
remove package logs
Args:
package_base(str): package base
version(str | None): package version to remove logs. If ``None`` is set, all logs will be removed
"""
self.client.package_logs_remove(package_base, version)
async def package_patches_get(self, package_base: str, variable: str | None) -> list[PkgbuildPatch]:
"""
get package patches
Args:
package_base(str): package base to retrieve
variable(str | None): optional filter by patch variable
Returns:
list[PkgbuildPatch]: list of patches for the specified package
"""
return self.client.package_patches_get(package_base, variable)
async def package_patches_remove(self, package_base: str, variable: str | None) -> None:
"""
remove package patch
Args:
package_base(str): package base to update
variable(str | None): patch name. If ``None`` is set, all patches will be removed
"""
self.client.package_patches_remove(package_base, variable)
async def package_patches_update(self, package_base: str, patch: PkgbuildPatch) -> None:
"""
create or update package patch
Args:
package_base(str): package base to update
patch(PkgbuildPatch): package patch
"""
self.client.package_patches_update(package_base, patch)
async def package_remove(self, package_base: str) -> None:
"""
remove package base from known list if any
Args:
package_base(str): package base
"""
async with self._lock:
self._known.pop(package_base, None)
self.client.package_remove(package_base)
await self.event_bus.broadcast(EventType.PackageRemoved, package_base)
async def package_status_update(self, package_base: str, status: BuildStatusEnum) -> None:
"""
update package status
Args:
package_base(str): package base to update
status(BuildStatusEnum): new build status
"""
package, current_status = await self.package_get(package_base)
async with self._lock:
self._known[package_base] = (package, BuildStatus(status, is_held=current_status.is_held))
self.client.package_status_update(package_base, status)
await self.event_bus.broadcast(EventType.PackageStatusChanged, package_base, status=status.value)
async def package_update(self, package: Package, status: BuildStatusEnum) -> None:
"""
update package
Args:
package(Package): package description
status(BuildStatusEnum): new build status
"""
async with self._lock:
_, current_status = self._known.get(package.base, (package, BuildStatus()))
self._known[package.base] = (package, BuildStatus(status, is_held=current_status.is_held))
self.client.package_update(package, status)
await self.event_bus.broadcast(
EventType.PackageUpdated, package.base, status=status.value, version=package.version,
)
async def packages(self) -> list[tuple[Package, BuildStatus]]:
"""
get current known packages list
Returns:
list[tuple[Package, BuildStatus]]: list of packages together with their statuses
"""
async with self._lock:
return list(self._known.values())
async def shutdown(self) -> None:
"""
gracefully shutdown watcher
"""
await self.event_bus.shutdown()
async def status_update(self, status: BuildStatusEnum) -> None:
"""
update service status
Args:
status(BuildStatusEnum): new service status
"""
self.status = BuildStatus(status)
await self.event_bus.broadcast(EventType.ServiceStatusChanged, None, status=status.value)
def __call__(self, package_base: str | None) -> Self:
"""
extract client for future calls
Args:
package_base(str | None): package base to validate that package exists if applicable
Returns:
Self: instance of self to pass calls to the client
Raises:
UnknownPackageError: if no package found
"""
# keep check here instead of calling package_get to keep this method synchronized
if package_base is not None and package_base not in self._known:
raise UnknownPackageError(package_base)
return self
@@ -1,507 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# pylint: disable=too-many-public-methods
import contextlib
from urllib.parse import quote_plus as url_encode
from ahriman.core.configuration import Configuration
from ahriman.core.http import SyncAhrimanClient
from ahriman.core.status import Client
from ahriman.models.build_status import BuildStatus, BuildStatusEnum
from ahriman.models.changes import Changes
from ahriman.models.dependencies import Dependencies
from ahriman.models.event import Event, EventType
from ahriman.models.internal_status import InternalStatus
from ahriman.models.log_record import LogRecord
from ahriman.models.package import Package
from ahriman.models.pkgbuild_patch import PkgbuildPatch
from ahriman.models.repository_id import RepositoryId
class WebClient(Client, SyncAhrimanClient):
"""
build status reporter web client
Attributes:
repository_id(RepositoryId): repository unique identifier
"""
def __init__(self, repository_id: RepositoryId, configuration: Configuration) -> None:
"""
Args:
repository_id(RepositoryId): repository unique identifier
configuration(Configuration): configuration instance
"""
section, self.address = self.parse_address(configuration)
suppress_errors = configuration.getboolean( # read old-style first and then fallback to new style
"settings", "suppress_http_log_errors",
fallback=configuration.getboolean("status", "suppress_http_log_errors", fallback=False))
SyncAhrimanClient.__init__(self, configuration, section, suppress_errors=suppress_errors)
self.repository_id = repository_id
@staticmethod
def parse_address(configuration: Configuration) -> tuple[str, str]:
"""
parse address from legacy configuration
Args:
configuration(Configuration): configuration instance
Returns:
tuple[str, str]: tuple of section name and server address
"""
# new-style section
if (address := configuration.get("status", "address", fallback=None)) is not None:
return "status", address
# legacy-style section
if (unix_socket := configuration.get("web", "unix_socket", fallback=None)) is not None:
# special pseudo-protocol which is used for unix sockets
return "web", f"http+unix://{url_encode(unix_socket)}"
address = configuration.get("web", "address", fallback=None)
if not address:
# build address from host and port directly
host = configuration.get("web", "host")
port = configuration.getint("web", "port")
address = f"http://{host}:{port}"
return "web", address
def _changes_url(self, package_base: str) -> str:
"""
get url for the changes api
Args:
package_base(str): package base
Returns:
str: full url for web service for changes
"""
return f"{self.address}/api/v1/packages/{url_encode(package_base)}/changes"
def _dependencies_url(self, package_base: str) -> str:
"""
get url for the dependencies api
Args:
package_base(str): package base
Returns:
str: full url for web service for dependencies
"""
return f"{self.address}/api/v1/packages/{url_encode(package_base)}/dependencies"
def _events_url(self) -> str:
"""
get url for the events api
Returns:
str: full url for web service for events
"""
return f"{self.address}/api/v1/events"
def _logs_url(self, package_base: str) -> str:
"""
get url for the logs api
Args:
package_base(str): package base
Returns:
str: full url for web service for logs
"""
return f"{self.address}/api/v1/packages/{url_encode(package_base)}/logs"
def _package_url(self, package_base: str = "") -> str:
"""
package url generator
Args:
package_base(str, optional): package base to generate url (Default value = "")
Returns:
str: full url of web service for specific package base
"""
suffix = f"/{url_encode(package_base)}" if package_base else ""
return f"{self.address}/api/v1/packages{suffix}"
def _patches_url(self, package_base: str, variable: str = "") -> str:
"""
patches url generator
Args:
package_base(str): package base
variable(str, optional): patch variable name to generate url (Default value = "")
Returns:
str: full url of web service for the package patch
"""
suffix = f"/{url_encode(variable)}" if variable else ""
return f"{self.address}/api/v1/packages/{url_encode(package_base)}/patches{suffix}"
def _status_url(self) -> str:
"""
get url for the status api
Returns:
str: full url for web service for status
"""
return f"{self.address}/api/v1/status"
def configuration_reload(self) -> None:
"""
reload configuration
"""
with contextlib.suppress(Exception):
self.make_request("POST", f"{self.address}/api/v1/service/config")
def event_add(self, event: Event) -> None:
"""
create new event
Args:
event(Event): audit log event
"""
with contextlib.suppress(Exception):
self.make_request("POST", self._events_url(), params=self.repository_id.query(), json=event.view())
def event_get(self, event: str | EventType | None, object_id: str | None,
from_date: int | float | None = None, to_date: int | float | None = None,
limit: int = -1, offset: int = 0) -> list[Event]:
"""
retrieve list of events
Args:
event(str | EventType | None): filter by event type
object_id(str | None): filter by event object
from_date(int | float | None, optional): minimal creation date, inclusive (Default value = None)
to_date(int | float | None, optional): maximal creation date, exclusive (Default value = None)
limit(int, optional): limit records to the specified count, -1 means unlimited (Default value = -1)
offset(int, optional): records offset (Default value = 0)
Returns:
list[Event]: list of audit log events
"""
query = self.repository_id.query() + [("limit", str(limit)), ("offset", str(offset))]
if event is not None:
query.append(("event", str(event)))
if object_id is not None:
query.append(("object_id", object_id))
if from_date is not None:
query.append(("from_date", str(from_date)))
if to_date is not None:
query.append(("to_date", str(to_date)))
with contextlib.suppress(Exception):
response = self.make_request("GET", self._events_url(), params=query)
response_json = response.json()
return [Event.from_json(event) for event in response_json]
return []
def logs_rotate(self, keep_last_records: int) -> None:
"""
remove older logs from storage
Args:
keep_last_records(int): number of last records to keep
"""
query = self.repository_id.query() + [("keep_last_records", str(keep_last_records))]
with contextlib.suppress(Exception):
self.make_request("DELETE", f"{self.address}/api/v1/service/logs", params=query)
def package_changes_get(self, package_base: str) -> Changes:
"""
get package changes
Args:
package_base(str): package base to retrieve
Returns:
Changes: package changes if available and empty object otherwise
"""
with contextlib.suppress(Exception):
response = self.make_request("GET", self._changes_url(package_base),
params=self.repository_id.query())
response_json = response.json()
return Changes.from_json(response_json)
return Changes()
def package_changes_update(self, package_base: str, changes: Changes) -> None:
"""
update package changes
Args:
package_base(str): package base to update
changes(Changes): changes descriptor
"""
with contextlib.suppress(Exception):
self.make_request("POST", self._changes_url(package_base),
params=self.repository_id.query(), json=changes.view())
def package_dependencies_get(self, package_base: str) -> Dependencies:
"""
get package dependencies
Args:
package_base(str): package base to retrieve
Returns:
list[Dependencies]: package implicit dependencies if available
"""
with contextlib.suppress(Exception):
response = self.make_request("GET", self._dependencies_url(package_base),
params=self.repository_id.query())
response_json = response.json()
return Dependencies.from_json(response_json)
return Dependencies()
def package_dependencies_update(self, package_base: str, dependencies: Dependencies) -> None:
"""
update package dependencies
Args:
package_base(str): package base to update
dependencies(Dependencies): dependencies descriptor
"""
with contextlib.suppress(Exception):
self.make_request("POST", self._dependencies_url(package_base),
params=self.repository_id.query(), json=dependencies.view())
def package_get(self, package_base: str | None) -> list[tuple[Package, BuildStatus]]:
"""
get package status
Args:
package_base(str | None): package base to get
Returns:
list[tuple[Package, BuildStatus]]: list of current package description and status if it has been found
"""
with contextlib.suppress(Exception):
response = self.make_request("GET", self._package_url(package_base or ""),
params=self.repository_id.query())
response_json = response.json()
return [
(Package.from_json(package["package"]), BuildStatus.from_json(package["status"]))
for package in response_json
]
return []
def package_hold_update(self, package_base: str, *, enabled: bool) -> None:
"""
update package hold status
Args:
package_base(str): package base name
enabled(bool): new hold status
"""
with contextlib.suppress(Exception):
self.make_request("POST", f"{self.address}/api/v1/packages/{url_encode(package_base)}/hold",
params=self.repository_id.query(), json={"is_held": enabled})
def package_logs_add(self, log_record: LogRecord) -> None:
"""
post log record
Args:
log_record(LogRecord): log record
"""
payload = {
"created": log_record.created,
"message": log_record.message,
"process_id": log_record.log_record_id.process_id,
"version": log_record.log_record_id.version,
}
# this is special case, because we would like to do not suppress exception here
# in case of exception raised it will be handled by upstream HttpLogHandler
# In the other hand, we force to suppress all http logs here to avoid cyclic reporting
self.make_request("POST", self._logs_url(log_record.log_record_id.package_base),
params=self.repository_id.query(), json=payload, suppress_errors=True)
def package_logs_get(self, package_base: str, version: str | None = None, process_id: str | None = None,
limit: int = -1, offset: int = 0) -> list[LogRecord]:
"""
get package logs
Args:
package_base(str): package base
version(str | None, optional): package version to search (Default value = None)
process_id(str | None, optional): process identifier to search (Default value = None)
limit(int, optional): limit records to the specified count, -1 means unlimited (Default value = -1)
offset(int, optional): records offset (Default value = 0)
Returns:
list[LogRecord]: package logs
"""
query = self.repository_id.query() + [("limit", str(limit)), ("offset", str(offset))]
if version is not None:
query.append(("version", version))
if process_id is not None:
query.append(("process_id", process_id))
with contextlib.suppress(Exception):
response = self.make_request("GET", self._logs_url(package_base), params=query)
response_json = response.json()
return [LogRecord.from_json(package_base, record) for record in response_json]
return []
def package_logs_remove(self, package_base: str, version: str | None) -> None:
"""
remove package logs
Args:
package_base(str): package base
version(str | None): package version to remove logs. If ``None`` is set, all logs will be removed
"""
query = self.repository_id.query()
if version is not None:
query += [("version", version)]
with contextlib.suppress(Exception):
self.make_request("DELETE", self._logs_url(package_base), params=query)
def package_patches_get(self, package_base: str, variable: str | None) -> list[PkgbuildPatch]:
"""
get package patches
Args:
package_base(str): package base to retrieve
variable(str | None): optional filter by patch variable
Returns:
list[PkgbuildPatch]: list of patches for the specified package
"""
with contextlib.suppress(Exception):
response = self.make_request("GET", self._patches_url(package_base, variable or ""))
response_json = response.json()
patches = response_json if variable is None else [response_json]
return [PkgbuildPatch.from_json(patch) for patch in patches]
return []
def package_patches_remove(self, package_base: str, variable: str | None) -> None:
"""
remove package patch
Args:
package_base(str): package base to update
variable(str | None): patch name. If ``None`` is set, all patches will be removed
"""
with contextlib.suppress(Exception):
self.make_request("DELETE", self._patches_url(package_base, variable or ""))
def package_patches_update(self, package_base: str, patch: PkgbuildPatch) -> None:
"""
create or update package patch
Args:
package_base(str): package base to update
patch(PkgbuildPatch): package patch
"""
with contextlib.suppress(Exception):
self.make_request("POST", self._patches_url(package_base), json=patch.view())
def package_remove(self, package_base: str) -> None:
"""
remove packages from watcher
Args:
package_base(str): basename to remove
"""
with contextlib.suppress(Exception):
self.make_request("DELETE", self._package_url(package_base), params=self.repository_id.query())
def package_status_update(self, package_base: str, status: BuildStatusEnum) -> None:
"""
update package build status. Unlike :func:`package_update()` it does not update package properties
Args:
package_base(str): package base to update
status(BuildStatusEnum): current package build status
Raises:
NotImplementedError: not implemented method
"""
payload = {"status": status.value}
with contextlib.suppress(Exception):
self.make_request("POST", self._package_url(package_base),
params=self.repository_id.query(), json=payload)
def package_update(self, package: Package, status: BuildStatusEnum) -> None:
"""
add new package or update existing one with status
Args:
package(Package): package properties
status(BuildStatusEnum): current package build status
Raises:
NotImplementedError: not implemented method
"""
payload = {
"status": status.value,
"package": package.view(),
}
with contextlib.suppress(Exception):
self.make_request("POST", self._package_url(package.base),
params=self.repository_id.query(), json=payload)
def status_get(self) -> InternalStatus:
"""
get internal service status
Returns:
InternalStatus: current internal (web) service status
"""
with contextlib.suppress(Exception):
response = self.make_request("GET", self._status_url(), params=self.repository_id.query())
response_json = response.json()
return InternalStatus.from_json(response_json)
return InternalStatus(status=BuildStatus())
def status_update(self, status: BuildStatusEnum) -> None:
"""
update ahriman status itself
Args:
status(BuildStatusEnum): current ahriman status
"""
payload = {"status": status.value}
with contextlib.suppress(Exception):
self.make_request("POST", self._status_url(), params=self.repository_id.query(), json=payload)
-47
View File
@@ -1,47 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from typing import Any, Protocol
class Comparable(Protocol):
"""
class which supports :func:`__lt__` operation`
"""
def __lt__(self, other: Any) -> bool: ...
class HasBool(Protocol):
"""
class which defines :func:`bool()` method
"""
def __bool__(self) -> bool: ...
class HasLength(Protocol):
"""
class which defines :func:`len()` method
"""
def __len__(self) -> int: ...
ExplicitBool = HasBool | HasLength | int
-21
View File
@@ -1,21 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# backward compatibility wrapper
from ahriman.core.utils import * # pylint: disable=wildcard-import,unused-wildcard-import
@@ -1,19 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
@@ -1,66 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from dataclasses import dataclass, field, fields
from typing import Any, Self
from ahriman.core.utils import dataclass_view, filter_json
@dataclass(frozen=True)
class Dependencies:
"""
package paths dependencies
Attributes:
paths(dict[str, list[str]]): map of the paths used by this package to set of packages in which they were found
"""
paths: dict[str, list[str]] = field(default_factory=dict)
def __post_init__(self) -> None:
"""
remove empty paths
"""
paths = {path: packages for path, packages in self.paths.items() if packages}
object.__setattr__(self, "paths", paths)
@classmethod
def from_json(cls, dump: dict[str, Any]) -> Self:
"""
construct dependencies from the json dump
Args:
dump(dict[str, Any]): json dump body
Returns:
Self: dependencies object
"""
# filter to only known fields
known_fields = [pair.name for pair in fields(cls)]
return cls(**filter_json(dump, known_fields))
def view(self) -> dict[str, Any]:
"""
generate json dependencies view
Returns:
dict[str, Any]: json-friendly dictionary
"""
return dataclass_view(self)
-146
View File
@@ -1,146 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from enum import StrEnum
from typing import Any, Self
from ahriman.core.utils import utcnow
class EventType(StrEnum):
"""
predefined event types
Attributes:
BuildLog(EventType): new build log line
PackageHeld(EventType): package hold status has been changed
PackageOutdated(EventType): package has been marked as out-of-date
PackageRemoved(EventType): package has been removed
PackageStatusChanged(EventType): package build status has been changed
PackageUpdateFailed(EventType): package update has been failed
PackageUpdated(EventType): package has been updated
ServiceStatusChanged(EventType): service status has been changed
"""
BuildLog = "build-log"
PackageHeld = "package-held"
PackageOutdated = "package-outdated"
PackageRemoved = "package-removed"
PackageStatusChanged = "package-status-changed"
PackageUpdateFailed = "package-update-failed"
PackageUpdated = "package-updated"
ServiceStatusChanged = "service-status-changed"
class Event:
"""
audit log event
Attributes:
created(int): event timestamp
data(dict[str, Any]): event metadata
event(str | EventType): event type
message(str | None): event message if available
object_id(str): object identifier
"""
def __init__(self, event: str | EventType, object_id: str, message: str | None = None, created: int | None = None,
**kwargs: Any):
"""
Args:
event(str | EventType): event type
object_id(str): object identifier
message(str | None): event message if available
created(int | None, optional): event timestamp (Default value = None)
**kwargs(Any): event metadata
"""
self.event = EventType(event) if event in EventType else event
self.object_id = object_id
self.created = created or int(utcnow().timestamp())
self.message = message
self.data = kwargs
@classmethod
def from_json(cls, dump: dict[str, Any]) -> Self:
"""
construct event from the JSON dump
Args:
dump(dict[str, Any]): JSON dump body
Returns:
Self: event object
"""
return cls(
event=dump["event"],
object_id=dump["object_id"],
message=dump.get("message"),
created=dump.get("created"),
**dump.get("data", {}),
)
def get(self, key: str) -> Any:
"""
get a property
Args:
key(str): key to lookup in data
Returns:
Any: metadata property if available or ``None`` otherwise
"""
return self.data.get(key)
def view(self) -> dict[str, Any]:
"""
generate JSON event view
Returns:
dict[str, Any]: json-friendly dictionary
"""
dump = {
"event": self.event,
"object_id": self.object_id,
"created": self.created,
}
if self.message is not None:
dump["message"] = self.message
if self.data:
dump["data"] = self.data
return dump
def __eq__(self, other: Any) -> bool:
"""
check if other is the same object
Args:
other(Any): other object instance
Returns:
bool: ``True`` if the other object is the same and ``False`` otherwise
"""
if not isinstance(other, Event):
return False
return self.event == other.event \
and self.object_id == other.object_id \
and self.message == other.message \
and self.created == other.created \
and self.data == other.data
@@ -1,90 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from __future__ import annotations
from collections.abc import Iterable
from dataclasses import dataclass, field
from pathlib import Path
from ahriman.core.utils import trim_package
@dataclass(frozen=True, kw_only=True)
class FilesystemPackage:
"""
class representing a simplified model for the package installed to filesystem
Attributes:
package_name(str): package name
depends(set[str]): list of package dependencies
directories(set[Path]): list of directories this package contains
files(list[Path]): list of files this package contains
opt_depends(set[str]): list of package optional dependencies
"""
package_name: str
depends: set[str]
opt_depends: set[str]
directories: list[Path] = field(default_factory=list)
files: list[Path] = field(default_factory=list)
def __post_init__(self) -> None:
"""
update dependencies list accordingly
"""
object.__setattr__(self, "depends", {trim_package(package) for package in self.depends})
object.__setattr__(self, "opt_depends", {trim_package(package) for package in self.opt_depends})
def depends_on(self, package_name: str, *, include_optional: bool) -> bool:
"""
check if package depends on given package name
Args:
package_name(str): package name to check dependencies
include_optional(bool): include optional dependencies to check
Returns:
bool: ``True`` in case if the given package in the dependencies lists
"""
if package_name in self.depends:
return True
if include_optional and package_name in self.opt_depends:
return True
return False
def is_root_package(self, packages: Iterable[FilesystemPackage], *, include_optional: bool) -> bool:
"""
check if the package is the one of the root packages. This method checks if there are any packages which are
dependency of the package and - to avoid circular dependencies - does not depend on the package. In addition,
if ``include_optional`` is set to ``True``, then it will also check optional dependencies of the package
Args:
packages(Iterable[FilesystemPackage]): list of packages in which we need to search
include_optional(bool): include optional dependencies to check
Returns:
bool: whether this package depends on any other package in the list of packages
"""
return not any(
package
for package in packages
if self.depends_on(package.package_name, include_optional=include_optional)
and not package.depends_on(self.package_name, include_optional=False)
)
@@ -1,76 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from dataclasses import dataclass
from typing import Any, Self
from ahriman.models.log_record_id import LogRecordId
@dataclass(frozen=True)
class LogRecord:
"""
log record
Attributes:
log_record_id(LogRecordId): log record identifier
created(float): log record creation timestamp
message(str): log record message
"""
log_record_id: LogRecordId
created: float
message: str
@classmethod
def from_json(cls, package_base: str, dump: dict[str, Any]) -> Self:
"""
construct log record from the JSON dump
Args:
package_base(str): package base for which log record belongs
dump(dict[str, Any]): JSON dump body
Returns:
Self: log record object
"""
if "process_id" in dump:
log_record_id = LogRecordId(package_base, dump["version"], dump["process_id"])
else:
log_record_id = LogRecordId(package_base, dump["version"])
return cls(
log_record_id=log_record_id,
created=dump["created"],
message=dump["message"],
)
def view(self) -> dict[str, Any]:
"""
generate JSON log record view
Returns:
dict[str, Any]: json-friendly dictionary
"""
return {
"created": self.created,
"message": self.message,
"version": self.log_record_id.version,
"process_id": self.log_record_id.process_id,
}
@@ -1,49 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import uuid
from dataclasses import dataclass
from typing import ClassVar
@dataclass(frozen=True)
class LogRecordId:
"""
log record process identifier
Attributes:
DEFAULT_PROCESS_ID(str): (class attribute) default process identifier
package_base(str): package base for which log record belongs
version(str): package version for which log record belongs
process_id(str, optional): unique process identifier
"""
package_base: str
version: str
process_id: str = ""
DEFAULT_PROCESS_ID: ClassVar[str] = str(uuid.uuid4())
def __post_init__(self) -> None:
"""
assign process identifier from default if not set
"""
if not self.process_id:
object.__setattr__(self, "process_id", self.DEFAULT_PROCESS_ID)
@@ -1,91 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import time
from types import TracebackType
from typing import Literal, Self
from ahriman.core.exceptions import InitializeError
class MetricsTimer:
"""
metrics implementation
Attributes:
start_time(float | None): timer start time in monotonic time
Examples:
This class implements simple timer which allows to measure the time elapsed of the function. Usually it should
be used like::
>>> with MetricsTimer() as timer:
>>> do_something()
>>> print("Time elapsed for first function: %f", timer.elapsed)
>>> do_something_different()
>>> print("Time elapsed for all functions: %f", timer.elapsed)
"""
def __init__(self) -> None:
""""""
self.start_time: float | None = None
@property
def elapsed(self) -> float:
"""
get elapsed time since the start of the timer
Returns:
float: time elapsed in seconds
Raises:
InitializeError: in case if timer was not initialized correctly
"""
if self.start_time is None:
raise InitializeError("Timer must be started in the context manager")
stop_time = time.monotonic()
consumed_time_ms = int(1000 * (stop_time - self.start_time))
return consumed_time_ms / 1000
def __enter__(self) -> Self:
"""
start timer context
Returns:
Self: always instance of self
"""
self.start_time = time.monotonic()
return self
def __exit__(self, exc_type: type[Exception] | None, exc_val: Exception | None,
exc_tb: TracebackType) -> Literal[False]:
"""
finish timer context
Args:
exc_type(type[Exception] | None): exception type name if any
exc_val(Exception | None): exception raised if any
exc_tb(TracebackType): exception traceback if any
Returns:
Literal[False]: always ``False`` (do not suppress any exception)
"""
return False
-419
View File
@@ -1,419 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from __future__ import annotations
from collections.abc import Iterable
from dataclasses import dataclass
from pathlib import Path
from pyalpm import vercmp # type: ignore[import-not-found]
from typing import Any, Self
from ahriman.core.alpm.pacman import Pacman
from ahriman.core.alpm.pacman_handle import PacmanHandle
from ahriman.core.alpm.remote import AUR, Official, OfficialSyncdb
from ahriman.core.log import LazyLogging
from ahriman.core.utils import dataclass_view, full_version, list_flatmap, parse_version, srcinfo_property_list
from ahriman.models.package_description import PackageDescription
from ahriman.models.package_source import PackageSource
from ahriman.models.pkgbuild import Pkgbuild
from ahriman.models.remote_source import RemoteSource
@dataclass(kw_only=True)
class Package(LazyLogging):
"""
package properties representation
Attributes:
base(str): package base name
packager(str | None): package packager if available
packages(dict[str, PackageDescription): map of package names to their properties.
Filled only on load from archive
remote(RemoteSource): package remote source if applicable
version(str): package full version
Examples:
Different usages of this class may generate different (incomplete) data, e.g. if instantiating class from JSON::
>>> package = Package.from_json(dump)
it will contain every data available in the JSON body. Otherwise, if generate package from local archive::
>>> package = Package.from_archive(local_path, pacman)
it will probably miss file descriptions (in case if there are multiple packages which belong to the base).
The specific class load method must be defined based on the source provided. The following methods (mostly) must
be used: :func:`from_archive()`, :func:`from_aur()`, :func:`from_build()`, :func:`from_official()` for sources
:attr:`ahriman.models.package_source.PackageSource.Archive`,
:attr:`ahriman.models.package_source.PackageSource.AUR`,
:attr:`ahriman.models.package_source.PackageSource.Local` and
:attr:`ahriman.models.package_source.PackageSource.Repository` respectively:
>>> ahriman_package = Package.from_aur("ahriman")
>>> pacman_package = Package.from_official("pacman", pacman)
"""
base: str
version: str
remote: RemoteSource
packages: dict[str, PackageDescription]
packager: str | None = None
@property
def depends(self) -> list[str]:
"""
get package base dependencies
Returns:
list[str]: sum of dependencies per each package
"""
return list_flatmap(self.packages.values(), lambda package: package.depends)
@property
def depends_build(self) -> set[str]:
"""
get full list of external dependencies which has to be installed for build process
Returns:
set[str]: full dependencies list used by devtools
"""
return (set(self.depends) | set(self.depends_make) | set(self.depends_check)).difference(self.packages_full)
@property
def depends_check(self) -> list[str]:
"""
get package test dependencies
Returns:
list[str]: sum of test dependencies per each package
"""
return list_flatmap(self.packages.values(), lambda package: package.check_depends)
@property
def depends_make(self) -> list[str]:
"""
get package make dependencies
Returns:
list[str]: sum of make dependencies per each package
"""
return list_flatmap(self.packages.values(), lambda package: package.make_depends)
@property
def depends_opt(self) -> list[str]:
"""
get package optional dependencies
Returns:
list[str]: sum of optional dependencies per each package
"""
return list_flatmap(self.packages.values(), lambda package: package.opt_depends)
@property
def groups(self) -> list[str]:
"""
get package base groups
Returns:
list[str]: sum of groups per each package
"""
return list_flatmap(self.packages.values(), lambda package: package.groups)
@property
def is_vcs(self) -> bool:
"""
get VCS flag based on the package base
Returns:
bool: ``True`` in case if package base looks like VCS package and ``False`` otherwise
"""
return self.base.endswith("-bzr") \
or self.base.endswith("-cvs") \
or self.base.endswith("-darcs") \
or self.base.endswith("-git") \
or self.base.endswith("-hg") \
or self.base.endswith("-svn")
@property
def licenses(self) -> list[str]:
"""
get package base licenses
Returns:
list[str]: sum of licenses per each package
"""
return list_flatmap(self.packages.values(), lambda package: package.licenses)
@property
def packages_full(self) -> list[str]:
"""
get full packages list including provides
Returns:
list[str]: full list of packages which this base contains
"""
packages = set()
for package, properties in self.packages.items():
packages.add(package)
packages.update(properties.provides)
return sorted(packages)
@classmethod
def from_archive(cls, path: Path) -> Self:
"""
construct package properties from package archive
Args:
path(Path): path to package archive
Returns:
Self: package properties
"""
package = PacmanHandle.ephemeral().package_load(path)
description = PackageDescription.from_package(package, path)
return cls(
base=package.base or package.name,
version=package.version,
remote=RemoteSource(source=PackageSource.Archive),
packages={package.name: description},
packager=package.packager,
)
@classmethod
def from_aur(cls, name: str, packager: str | None = None, *, include_provides: bool = False) -> Self:
"""
construct package properties from AUR page
Args:
name(str): package name (either base or normal name)
packager(str | None, optional): packager to be used for this build (Default value = None)
include_provides(bool, optional): search by provides if no exact match found (Default value = False)
Returns:
Self: package properties
"""
package = AUR.info(name, include_provides=include_provides)
remote = RemoteSource(
source=PackageSource.AUR,
git_url=AUR.remote_git_url(package.package_base, package.repository),
web_url=AUR.remote_web_url(package.package_base),
path=".",
branch="master",
)
return cls(
base=package.package_base,
version=package.version,
remote=remote,
packages={package.name: PackageDescription.from_aur(package)},
packager=packager,
)
@classmethod
def from_build(cls, path: Path, architecture: str, packager: str | None = None) -> Self:
"""
construct package properties from sources directory
Args:
path(Path): path to package sources directory
architecture(str): load package for specific architecture
packager(str | None, optional): packager to be used for this build (Default value = None)
Returns:
Self: package properties
"""
pkgbuild = Pkgbuild.from_file(path / "PKGBUILD")
packages = {
package: PackageDescription(
depends=srcinfo_property_list("depends", pkgbuild, properties, architecture=architecture),
make_depends=srcinfo_property_list("makedepends", pkgbuild, properties, architecture=architecture),
opt_depends=srcinfo_property_list("optdepends", pkgbuild, properties, architecture=architecture),
check_depends=srcinfo_property_list("checkdepends", pkgbuild, properties, architecture=architecture),
)
for package, properties in pkgbuild.packages().items()
}
version = full_version(pkgbuild.get("epoch"), pkgbuild["pkgver"], pkgbuild["pkgrel"])
remote = RemoteSource(
source=PackageSource.Local,
git_url=path.absolute().as_uri(),
web_url=None,
path=".",
branch="master",
)
return cls(
base=pkgbuild["pkgbase"],
version=version,
remote=remote,
packages=packages,
packager=packager,
)
@classmethod
def from_json(cls, dump: dict[str, Any]) -> Self:
"""
construct package properties from JSON dump
Args:
dump(dict[str, Any]): JSON dump body
Returns:
Self: package properties
"""
packages_json = dump.get("packages") or {}
packages = {
key: PackageDescription.from_json(value)
for key, value in packages_json.items()
}
remote = dump.get("remote") or {}
return cls(
base=dump["base"],
version=dump["version"],
remote=RemoteSource.from_json(remote),
packages=packages,
packager=dump.get("packager"),
)
@classmethod
def from_official(cls, name: str, pacman: Pacman, packager: str | None = None, *, use_syncdb: bool = True,
include_provides: bool = False) -> Self:
"""
construct package properties from official repository page
Args:
name(str): package name (either base or normal name)
pacman(Pacman): alpm wrapper instance
packager(str | None, optional): packager to be used for this build (Default value = None)
use_syncdb(bool, optional): use pacman databases instead of official repositories RPC (Default value = True)
include_provides(bool, optional): search by provides if no exact match found (Default value = False)
Returns:
Self: package properties
"""
impl = OfficialSyncdb if use_syncdb else Official
package = impl.info(name, pacman=pacman, include_provides=include_provides)
remote = RemoteSource(
source=PackageSource.Repository,
git_url=Official.remote_git_url(package.package_base, package.repository),
web_url=Official.remote_web_url(package.package_base),
path=".",
branch="main",
)
return cls(
base=package.package_base,
version=package.version,
remote=remote,
packages={package.name: PackageDescription.from_aur(package)},
packager=packager,
)
def next_pkgrel(self, local_version: str | None) -> str | None:
"""
generate next pkgrel variable. The package release will be incremented if ``local_version`` is more or equal to
the :attr:`version`; in this case the function will return new pkgrel value, otherwise ``None`` will be
returned
Args:
local_version(str | None): locally stored package version if available
Returns:
str | None: new generated package release version if any. In case if the release contains dot (e.g. 1.2),
the minor part will be incremented by 1. If the release does not contain major.minor notation, the minor
version equals to 1 will be appended
"""
if local_version is None:
return None # local version not found, keep upstream pkgrel
if self.vercmp(local_version) > 0:
return None # upstream version is newer than local one, keep upstream pkgrel
*_, local_pkgrel = parse_version(local_version)
if "." in local_pkgrel:
major, minor = local_pkgrel.rsplit(".", maxsplit=1)
else:
major, minor = local_pkgrel, "0"
return f"{major}.{int(minor) + 1}"
def pretty_print(self) -> str:
"""
generate pretty string representation
Returns:
str: print-friendly string
"""
is_single_package = self.base in self.packages and len(self.packages) == 1
details = "" if is_single_package else f" ({" ".join(sorted(self.packages.keys()))})"
return f"{self.base}{details}"
def supports_architecture(self, architecture: str) -> bool:
"""
helper to check if the package belongs to the specified architecture
Args:
architecture(str): probe repository architecture
Returns:
bool: ``True`` if all packages are same architecture or any
"""
return all(single.architecture in ("any", architecture) for single in self.packages.values())
def vercmp(self, version: str) -> int:
"""
typed wrapper around :func:`pyalpm.vercmp()`
Args:
version(str): version to compare
Returns:
int: negative if current version is less than provided, positive if greater than and zero if equals
"""
result: int = vercmp(self.version, version)
return result
def view(self) -> dict[str, Any]:
"""
generate JSON package view
Returns:
dict[str, Any]: json-friendly dictionary
"""
return dataclass_view(self)
def with_packages(self, packages: Iterable[Path]) -> None:
"""
replace packages descriptions with ones from archives
Args:
packages(Iterable[Path]): paths to package archives
"""
self.packages = {} # reset state
for package in packages:
archive = self.from_archive(package)
if archive.base != self.base:
continue
self.packages.update(archive.packages)
-213
View File
@@ -1,213 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from collections.abc import Iterator, Mapping
from dataclasses import dataclass
from io import StringIO
from pathlib import Path
from typing import Any, ClassVar, IO, Self
from urllib.parse import urlparse
from ahriman.core.alpm.pkgbuild_parser import PkgbuildParser, PkgbuildToken
from ahriman.core.utils import srcinfo_property_list
from ahriman.models.pkgbuild_patch import PkgbuildPatch
@dataclass(frozen=True)
class Pkgbuild(Mapping[str, Any]):
"""
model and proxy for PKGBUILD properties
Attributes:
DEFAULT_ENCODINGS(str): (class attribute) default encoding to be applied on the file content
fields(dict[str, PkgbuildPatch]): PKGBUILD fields
"""
fields: dict[str, PkgbuildPatch]
DEFAULT_ENCODINGS: ClassVar[str] = "utf8"
@property
def variables(self) -> dict[str, str]:
"""
list of variables defined and (maybe) used in this PKGBUILD
Returns:
dict[str, str]: map of variable name to its value. The value will be included here in case if it presented
in the internal dictionary, it is not a function and the value has string type
"""
return {
key: value.value
for key, value in self.fields.items()
if not value.is_function and isinstance(value.value, str)
}
@classmethod
def from_file(cls, path: Path, encoding: str | None = None) -> Self:
"""
parse PKGBUILD from the file
Args:
path(Path): path to the PKGBUILD file
encoding(str | None, optional): the encoding of the file (Default value = None)
Returns:
Self: constructed instance of self
Raises:
EncodeError: if encoding is unknown
"""
# read raw bytes from file
with path.open("rb") as input_file:
content = input_file.read()
# decode bytes content based on either
encoding = encoding or cls.DEFAULT_ENCODINGS
io = StringIO(content.decode(encoding, errors="backslashreplace"))
return cls.from_io(io)
@classmethod
def from_io(cls, stream: IO[str]) -> Self:
"""
parse PKGBUILD from input stream
Args:
stream(IO[str]): input stream containing PKGBUILD content
Returns:
Self: constructed instance of self
"""
parser = PkgbuildParser(stream)
fields = {patch.key: patch for patch in parser.parse()}
# pkgbase is optional field, the pkgname must be used instead if not set
# however, pkgname is not presented is "package()" functions which we are parsing here too,
# thus, in our terms, it is optional too
if "pkgbase" not in fields and "pkgname" in fields:
fields["pkgbase"] = PkgbuildPatch("pkgbase", fields["pkgname"].value)
return cls({key: value for key, value in fields.items() if key})
@staticmethod
def local_files(path: Path) -> Iterator[Path]:
"""
extract list of local files
Args:
path(Path): path to package sources directory
Yields:
Path: list of paths of files which belong to the package and distributed together with this tarball.
All paths are relative to the ``path``
Raises:
PackageInfoError: if there are parsing errors
"""
pkgbuild = Pkgbuild.from_file(path / "PKGBUILD")
# we could use arch property, but for consistency it is better to call special method
architectures = Pkgbuild.supported_architectures(path)
for architecture in architectures:
for source in srcinfo_property_list("source", pkgbuild, {}, architecture=architecture):
if "::" in source:
_, source = source.split("::", maxsplit=1) # in case if filename is specified, remove it
if urlparse(source).scheme:
# basically file schema should use absolute path which is impossible if we are distributing
# files together with PKGBUILD. In this case we are going to skip it also
continue
yield Path(source)
if (install := pkgbuild.get("install")) is not None:
yield Path(install)
@staticmethod
def supported_architectures(path: Path) -> set[str]:
"""
load supported architectures from package sources
Args:
path(Path): path to package sources directory
Returns:
set[str]: list of package supported architectures
"""
pkgbuild = Pkgbuild.from_file(path / "PKGBUILD")
return set(pkgbuild.get("arch", []))
def packages(self) -> dict[str, Self]:
"""
extract properties from internal package functions
Returns:
dict[str, Self]: map of package name to its inner properties if defined
"""
packages = [self["pkgname"]] if isinstance(self["pkgname"], str) else self["pkgname"]
def io(package_name: str) -> IO[str]:
# try to read package specific function and fallback to default otherwise
content = self.get(f"package_{package_name}") or self.get("package") or ""
return StringIO(content)
return {package: self.from_io(io(package)) for package in packages}
def __getitem__(self, item: str) -> Any:
"""
get the field of the PKGBUILD. This method tries to get exact key value if possible; if none was found,
it tries to fetch function with the same name
Args:
item(str): key name
Returns:
Any: substituted value by the key
Raises:
KeyError: if key doesn't exist
"""
value = self.fields.get(item)
# if the key wasn't found and user didn't ask for function explicitly, we can try to get by function name
if value is None and not item.endswith(PkgbuildToken.FunctionDeclaration):
value = self.fields.get(f"{item}{PkgbuildToken.FunctionDeclaration}")
# if we still didn't find anything, we can just raise the exception
if value is None:
raise KeyError(item)
return value.substitute(self.variables)
def __iter__(self) -> Iterator[str]:
"""
iterate over the fields
Returns:
Iterator[str]: keys iterator
"""
return iter(self.fields)
def __len__(self) -> int:
"""
get length of the mapping
Returns:
int: amount of the fields in this PKGBUILD
"""
return len(self.fields)
@@ -1,77 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from dataclasses import dataclass, fields
from typing import Any, Self
from ahriman.core.utils import filter_json
from ahriman.models.package import Package
@dataclass(frozen=True, kw_only=True)
class RepositoryStats:
"""
repository stats representation
"""
bases: int
packages: int
archive_size: int
installed_size: int
@classmethod
def from_json(cls, dump: dict[str, Any]) -> Self:
"""
construct counters from JSON dump
Args:
dump(dict[str, Any]): JSON dump body
Returns:
Self: status counters
"""
# filter to only known fields
known_fields = [pair.name for pair in fields(cls)]
return cls(**filter_json(dump, known_fields))
@classmethod
def from_packages(cls, packages: list[Package]) -> Self:
"""
construct statistics from list of repository packages
Args:
packages(list[Packages]): list of repository packages
Returns:
Self: constructed statistics object
"""
return cls(
bases=len(packages),
packages=sum(len(package.packages) for package in packages),
archive_size=sum(
archive.archive_size or 0
for package in packages
for archive in package.packages.values()
),
installed_size=sum(
archive.installed_size or 0
for package in packages
for archive in package.packages.values()
),
)
@@ -1,59 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import re
from dataclasses import dataclass
from functools import cached_property
from pathlib import Path
@dataclass(frozen=True)
class ScanPaths:
"""
paths used for scan filesystem
Attributes:
paths(list[str]): list of regular expressions to be used to match paths
"""
paths: list[str]
@cached_property
def patterns(self) -> list[re.Pattern[str]]:
"""
compiled regular expressions
Returns:
list[re.Pattern]: a list of compiled regular expressions
"""
return [re.compile(path) for path in self.paths]
def is_allowed(self, path: Path) -> bool:
"""
check whether path is allowed to scan or not
Args:
path(Path): path to be checked
Returns:
bool: ``True`` in case if :attr:`paths` contains at least one element to which the path is matched
and ``False`` otherwise
"""
return any(pattern.match(str(path)) for pattern in self.patterns)
@@ -1,104 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import statistics
from dataclasses import dataclass
@dataclass(frozen=True)
class SeriesStatistics:
"""
series statistics helper
Attributes:
series(list[float | int]): list of values to be processed
"""
series: list[float | int]
@property
def max(self) -> float | int | None:
"""
get max value in series
Returns:
float | int | None: ``None`` if series is empty and maximal value otherwise``
"""
if self:
return max(self.series)
return None
@property
def mean(self) -> float | int | None:
"""
get mean value in series
Returns:
float | int | None: ``None`` if series is empty and mean value otherwise
"""
if self:
return statistics.mean(self.series)
return None
@property
def min(self) -> float | int | None:
"""
get min value in series
Returns:
float | int | None: ``None`` if series is empty and minimal value otherwise
"""
if self:
return min(self.series)
return None
@property
def st_dev(self) -> float | None:
"""
get standard deviation in series
Returns:
float | None: ``None`` if series size is less than 1, 0 if series contains single element and standard
deviation otherwise
"""
if not self:
return None
if len(self.series) > 1:
return statistics.stdev(self.series)
return 0.0
@property
def total(self) -> int:
"""
retrieve amount of elements
Returns:
int: the series collection size
"""
return len(self.series)
def __bool__(self) -> bool:
"""
check if series is empty or not
Returns:
bool: ``True`` if series contains elements and ``False`` otherwise
"""
return bool(self.total)
-138
View File
@@ -1,138 +0,0 @@
#
# Copyright (c) 2021-2026 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import time
from collections.abc import Callable
from dataclasses import dataclass, field
from typing import Literal, ParamSpec
from ahriman.models.metrics_timer import MetricsTimer
Params = ParamSpec("Params")
@dataclass(frozen=True)
class WaiterResult:
"""
representation of a waiter result. This class should not be used directly, use derivatives instead
Attributes:
took(float): consumed time in seconds
"""
took: float
def __bool__(self) -> bool:
"""
indicates whether the waiter completed with success or not
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def __float__(self) -> float:
"""
extract time spent to retrieve the result in seconds
Returns:
float: consumed time in seconds
"""
return self.took
class WaiterTaskFinished(WaiterResult):
"""
a waiter result used to notify that the task has been completed successfully
"""
def __bool__(self) -> Literal[True]:
"""
indicates whether the waiter completed with success or not
Returns:
Literal[True]: always ``True``
"""
return True
class WaiterTimedOut(WaiterResult):
"""
a waiter result used to notify that the waiter run out of time
"""
def __bool__(self) -> Literal[False]:
"""
indicates whether the waiter completed with success or not
Returns:
Literal[False]: always ``False``
"""
return False
@dataclass(frozen=True)
class Waiter:
"""
simple waiter implementation
Attributes:
interval(float): interval in seconds between checks
wait_timeout(float): timeout in seconds to wait for. Negative value will result in immediate exit. Zero value
means infinite timeout
"""
wait_timeout: float
interval: float = field(default=10, kw_only=True)
def is_timed_out(self, elapsed: float) -> bool:
"""
check if timer is out
Attributes:
elapsed(float): elapsed time in seconds
Returns:
bool: ``True`` in case current monotonic time is more than :attr:`start_time` and :attr:`wait_timeout`
doesn't equal to 0
"""
return self.wait_timeout != 0 and elapsed > self.wait_timeout
def wait(self, in_progress: Callable[Params, bool], *args: Params.args, **kwargs: Params.kwargs) -> WaiterResult:
"""
wait until requirements are not met
Args:
in_progress(Callable[Params, bool]): function to check if timer should wait for another cycle
*args(Params.args): positional arguments for check call
**kwargs(Params.kwargs): keyword arguments for check call
Returns:
WaiterResult: waiter result object
"""
with MetricsTimer() as timer:
while not (timed_out := self.is_timed_out(timer.elapsed)) and in_progress(*args, **kwargs):
time.sleep(self.interval)
took = timer.elapsed
if timed_out:
return WaiterTimedOut(took)
return WaiterTaskFinished(took)
@@ -1,85 +0,0 @@
import argparse
import pytest
from pytest_mock import MockerFixture
from ahriman.application.application import Application
from ahriman.application.handlers.add import Add
from ahriman.core.configuration import Configuration
from ahriman.core.repository import Repository
from ahriman.models.package_source import PackageSource
from ahriman.models.pkgbuild_patch import PkgbuildPatch
def _default_args(args: argparse.Namespace) -> argparse.Namespace:
"""
default arguments for these test cases
Args:
args(argparse.Namespace): command line arguments fixture
Returns:
argparse.Namespace: generated arguments for these test cases
"""
args.package = ["ahriman"]
args.now = False
args.refresh = 0
args.source = PackageSource.Auto
args.username = "username"
args.variable = None
return args
def test_run(args: argparse.Namespace, configuration: Configuration, repository: Repository,
mocker: MockerFixture) -> None:
"""
must run command
"""
args = _default_args(args)
mocker.patch("ahriman.core.repository.Repository.load", return_value=repository)
on_start_mock = mocker.patch("ahriman.application.application.Application.on_start")
perform_mock = mocker.patch("ahriman.application.handlers.add.Add.perform_action")
_, repository_id = configuration.check_loaded()
Add.run(args, repository_id, configuration, report=False)
on_start_mock.assert_called_once_with()
perform_mock.assert_called_once_with(pytest.helpers.anyvar(int), args)
def test_perform_action(args: argparse.Namespace, application: Application, mocker: MockerFixture) -> None:
"""
must perform add action
"""
args = _default_args(args)
application_mock = mocker.patch("ahriman.application.application.Application.add")
update_mock = mocker.patch("ahriman.application.handlers.update.Update.perform_action")
Add.perform_action(application, args)
application_mock.assert_called_once_with(args.package, args.source, args.username)
update_mock.assert_not_called()
def test_perform_action_with_patches(args: argparse.Namespace, application: Application, mocker: MockerFixture) -> None:
"""
must perform add action and insert temporary patches
"""
args = _default_args(args)
args.variable = ["KEY=VALUE"]
mocker.patch("ahriman.application.application.Application.add")
patches_mock = mocker.patch("ahriman.core.status.local_client.LocalClient.package_patches_update")
Add.perform_action(application, args)
patches_mock.assert_called_once_with(args.package[0], PkgbuildPatch("KEY", "VALUE"))
def test_perform_action_with_updates(args: argparse.Namespace, application: Application, mocker: MockerFixture) -> None:
"""
must perform add action with updates after
"""
args = _default_args(args)
args.now = True
mocker.patch("ahriman.application.application.Application.add")
update_mock = mocker.patch("ahriman.application.handlers.update.Update.perform_action")
Add.perform_action(application, args)
update_mock.assert_called_once_with(application, args)
@@ -1,84 +0,0 @@
import argparse
import pytest
from pytest_mock import MockerFixture
from ahriman.application.handlers.archives import Archives
from ahriman.core.configuration import Configuration
from ahriman.core.database import SQLite
from ahriman.core.repository import Repository
from ahriman.models.action import Action
from ahriman.models.package import Package
def _default_args(args: argparse.Namespace) -> argparse.Namespace:
"""
default arguments for these test cases
Args:
args(argparse.Namespace): command line arguments fixture
Returns:
argparse.Namespace: generated arguments for these test cases
"""
args.action = Action.List
args.exit_code = False
args.info = False
args.package = "package"
return args
def test_run(args: argparse.Namespace, configuration: Configuration, repository: Repository,
package_ahriman: Package, mocker: MockerFixture) -> None:
"""
must run command
"""
args = _default_args(args)
mocker.patch("ahriman.core.repository.Repository.load", return_value=repository)
application_mock = mocker.patch("ahriman.core.repository.package_info.PackageInfo.package_archives",
return_value=[package_ahriman])
check_mock = mocker.patch("ahriman.application.handlers.handler.Handler.check_status")
print_mock = mocker.patch("ahriman.core.formatters.Printer.print")
_, repository_id = configuration.check_loaded()
Archives.run(args, repository_id, configuration, report=False)
application_mock.assert_called_once_with(args.package)
check_mock.assert_called_once_with(False, True)
print_mock.assert_called_once_with(verbose=False, log_fn=pytest.helpers.anyvar(int), separator=": ")
def test_run_empty_exception(args: argparse.Namespace, configuration: Configuration, repository: Repository,
mocker: MockerFixture) -> None:
"""
must raise ExitCode exception on empty archives result
"""
args = _default_args(args)
args.exit_code = True
mocker.patch("ahriman.core.repository.Repository.load", return_value=repository)
mocker.patch("ahriman.core.repository.package_info.PackageInfo.package_archives", return_value=[])
check_mock = mocker.patch("ahriman.application.handlers.handler.Handler.check_status")
_, repository_id = configuration.check_loaded()
Archives.run(args, repository_id, configuration, report=False)
check_mock.assert_called_once_with(True, False)
def test_imply_with_report(args: argparse.Namespace, configuration: Configuration, database: SQLite,
mocker: MockerFixture) -> None:
"""
must create application object with native reporting
"""
args = _default_args(args)
mocker.patch("ahriman.core.database.SQLite.load", return_value=database)
load_mock = mocker.patch("ahriman.core.repository.Repository.load")
_, repository_id = configuration.check_loaded()
Archives.run(args, repository_id, configuration, report=False)
load_mock.assert_called_once_with(repository_id, configuration, database, report=True, refresh_pacman_database=0)
def test_disallow_multi_architecture_run() -> None:
"""
must not allow multi architecture run
"""
assert not Archives.ALLOW_MULTI_ARCHITECTURE_RUN
@@ -1,110 +0,0 @@
import argparse
import pytest
from pytest_mock import MockerFixture
from ahriman.application.application import Application
from ahriman.application.handlers.copy import Copy
from ahriman.core.configuration import Configuration
from ahriman.core.database import SQLite
from ahriman.core.repository import Repository
from ahriman.models.build_status import BuildStatusEnum
from ahriman.models.package import Package
from ahriman.models.package_source import PackageSource
def _default_args(args: argparse.Namespace) -> argparse.Namespace:
"""
default arguments for these test cases
Args:
args(argparse.Namespace): command line arguments fixture
Returns:
argparse.Namespace: generated arguments for these test cases
"""
args.source = "source"
args.package = ["ahriman"]
args.exit_code = False
args.remove = False
return args
def test_run(args: argparse.Namespace, configuration: Configuration, repository: Repository,
database: SQLite, package_ahriman: Package, mocker: MockerFixture) -> None:
"""
must run command
"""
args = _default_args(args)
mocker.patch("ahriman.core.database.SQLite.load", return_value=database)
mocker.patch("ahriman.core.repository.Repository.load", return_value=repository)
mocker.patch("ahriman.core.repository.Repository.packages", return_value=[package_ahriman])
application_mock = mocker.patch("ahriman.application.handlers.copy.Copy.copy_package")
update_mock = mocker.patch("ahriman.application.application.Application.update")
remove_mock = mocker.patch("ahriman.application.application.Application.remove")
on_start_mock = mocker.patch("ahriman.application.application.Application.on_start")
_, repository_id = configuration.check_loaded()
Copy.run(args, repository_id, configuration, report=False)
application_mock.assert_called_once_with(package_ahriman, pytest.helpers.anyvar(int), pytest.helpers.anyvar(int))
update_mock.assert_called_once_with([])
remove_mock.assert_not_called()
on_start_mock.assert_called_once_with()
def test_run_remove(args: argparse.Namespace, configuration: Configuration, repository: Repository,
database: SQLite, package_ahriman: Package, mocker: MockerFixture) -> None:
"""
must run command and remove packages afterward
"""
args = _default_args(args)
args.remove = True
mocker.patch("ahriman.core.database.SQLite.load", return_value=database)
mocker.patch("ahriman.core.repository.Repository.load", return_value=repository)
mocker.patch("ahriman.core.repository.Repository.packages", return_value=[package_ahriman])
mocker.patch("ahriman.application.handlers.copy.Copy.copy_package")
mocker.patch("ahriman.application.application.Application.update")
remove_mock = mocker.patch("ahriman.application.application.Application.remove")
_, repository_id = configuration.check_loaded()
Copy.run(args, repository_id, configuration, report=False)
remove_mock.assert_called_once_with(args.package)
def test_run_empty_exception(args: argparse.Namespace, configuration: Configuration, repository: Repository,
database: SQLite, mocker: MockerFixture) -> None:
"""
must raise ExitCode exception on empty result
"""
args = _default_args(args)
args.exit_code = True
mocker.patch("ahriman.core.database.SQLite.load", return_value=database)
mocker.patch("ahriman.core.repository.Repository.load", return_value=repository)
mocker.patch("ahriman.core.repository.Repository.packages", return_value=[])
mocker.patch("ahriman.application.application.Application.update")
check_mock = mocker.patch("ahriman.application.handlers.handler.Handler.check_status")
_, repository_id = configuration.check_loaded()
Copy.run(args, repository_id, configuration, report=False)
check_mock.assert_called_once_with(True, [])
def test_copy_package(package_ahriman: Package, application: Application, mocker: MockerFixture) -> None:
"""
must copy package between repositories and its metadata
"""
add_mock = mocker.patch("ahriman.application.application.Application.add")
changes_get_mock = mocker.patch("ahriman.core.status.local_client.LocalClient.package_changes_get")
changes_update_mock = mocker.patch("ahriman.core.status.local_client.LocalClient.package_changes_update")
deps_get_mock = mocker.patch("ahriman.core.status.local_client.LocalClient.package_dependencies_get")
deps_update_mock = mocker.patch("ahriman.core.status.local_client.LocalClient.package_dependencies_update")
package_update_mock = mocker.patch("ahriman.core.status.local_client.LocalClient.package_update")
path = application.repository.paths.repository / package_ahriman.packages[package_ahriman.base].filename
Copy.copy_package(package_ahriman, application, application)
add_mock.assert_called_once_with([str(path)], PackageSource.Archive)
changes_get_mock.assert_called_once_with(package_ahriman.base)
changes_update_mock.assert_called_once_with(package_ahriman.base, changes_get_mock.return_value)
deps_get_mock.assert_called_once_with(package_ahriman.base)
deps_update_mock.assert_called_once_with(package_ahriman.base, deps_get_mock.return_value)
package_update_mock.assert_called_once_with(package_ahriman, BuildStatusEnum.Pending)
@@ -1,53 +0,0 @@
import argparse
import pytest
from pytest_mock import MockerFixture
from ahriman.application.handlers.hold import Hold
from ahriman.core.configuration import Configuration
from ahriman.core.repository import Repository
from ahriman.models.action import Action
def _default_args(args: argparse.Namespace) -> argparse.Namespace:
"""
default arguments for these test cases
Args:
args(argparse.Namespace): command line arguments fixture
Returns:
argparse.Namespace: generated arguments for these test cases
"""
args.package = ["ahriman"]
return args
def test_run(args: argparse.Namespace, configuration: Configuration, repository: Repository,
mocker: MockerFixture) -> None:
"""
must run command
"""
args = _default_args(args)
args.action = Action.Update
mocker.patch("ahriman.core.repository.Repository.load", return_value=repository)
hold_mock = mocker.patch("ahriman.core.status.local_client.LocalClient.package_hold_update")
_, repository_id = configuration.check_loaded()
Hold.run(args, repository_id, configuration, report=False)
hold_mock.assert_called_once_with("ahriman", enabled=True)
def test_run_remove(args: argparse.Namespace, configuration: Configuration, repository: Repository,
mocker: MockerFixture) -> None:
"""
must remove held status
"""
args = _default_args(args)
args.action = Action.Remove
mocker.patch("ahriman.core.repository.Repository.load", return_value=repository)
hold_mock = mocker.patch("ahriman.core.status.local_client.LocalClient.package_hold_update")
_, repository_id = configuration.check_loaded()
Hold.run(args, repository_id, configuration, report=False)
hold_mock.assert_called_once_with("ahriman", enabled=False)
@@ -1,100 +0,0 @@
import argparse
import pytest
from pytest_mock import MockerFixture
from ahriman.application.handlers.pkgbuild import Pkgbuild
from ahriman.core.configuration import Configuration
from ahriman.core.database import SQLite
from ahriman.core.repository import Repository
from ahriman.models.action import Action
from ahriman.models.changes import Changes
def _default_args(args: argparse.Namespace) -> argparse.Namespace:
"""
default arguments for these test cases
Args:
args(argparse.Namespace): command line arguments fixture
Returns:
argparse.Namespace: generated arguments for these test cases
"""
args.action = Action.List
args.exit_code = False
args.package = "package"
return args
def test_run(args: argparse.Namespace, configuration: Configuration, repository: Repository,
mocker: MockerFixture) -> None:
"""
must run command
"""
args = _default_args(args)
mocker.patch("ahriman.core.repository.Repository.load", return_value=repository)
application_mock = mocker.patch("ahriman.core.status.local_client.LocalClient.package_changes_get",
return_value=Changes("sha", "change", "pkgbuild content"))
check_mock = mocker.patch("ahriman.application.handlers.handler.Handler.check_status")
print_mock = mocker.patch("ahriman.core.formatters.Printer.print")
_, repository_id = configuration.check_loaded()
Pkgbuild.run(args, repository_id, configuration, report=False)
application_mock.assert_called_once_with(args.package)
check_mock.assert_called_once_with(False, True)
print_mock.assert_called_once_with(verbose=True, log_fn=pytest.helpers.anyvar(int), separator="")
def test_run_empty_exception(args: argparse.Namespace, configuration: Configuration, repository: Repository,
mocker: MockerFixture) -> None:
"""
must raise ExitCode exception on empty pkgbuild result
"""
args = _default_args(args)
args.exit_code = True
mocker.patch("ahriman.core.repository.Repository.load", return_value=repository)
mocker.patch("ahriman.core.status.local_client.LocalClient.package_changes_get", return_value=Changes())
check_mock = mocker.patch("ahriman.application.handlers.handler.Handler.check_status")
_, repository_id = configuration.check_loaded()
Pkgbuild.run(args, repository_id, configuration, report=False)
check_mock.assert_called_once_with(True, False)
def test_run_remove(args: argparse.Namespace, configuration: Configuration, repository: Repository,
mocker: MockerFixture) -> None:
"""
must remove package pkgbuild
"""
args = _default_args(args)
args.action = Action.Remove
mocker.patch("ahriman.core.repository.Repository.load", return_value=repository)
changes = Changes("sha", "change", "pkgbuild content")
mocker.patch("ahriman.core.status.local_client.LocalClient.package_changes_get", return_value=changes)
update_mock = mocker.patch("ahriman.core.status.local_client.LocalClient.package_changes_update")
_, repository_id = configuration.check_loaded()
Pkgbuild.run(args, repository_id, configuration, report=False)
update_mock.assert_called_once_with(args.package, Changes("sha", "change", None))
def test_imply_with_report(args: argparse.Namespace, configuration: Configuration, database: SQLite,
mocker: MockerFixture) -> None:
"""
must create application object with native reporting
"""
args = _default_args(args)
mocker.patch("ahriman.core.database.SQLite.load", return_value=database)
load_mock = mocker.patch("ahriman.core.repository.Repository.load")
_, repository_id = configuration.check_loaded()
Pkgbuild.run(args, repository_id, configuration, report=False)
load_mock.assert_called_once_with(repository_id, configuration, database, report=True, refresh_pacman_database=0)
def test_disallow_multi_architecture_run() -> None:
"""
must not allow multi architecture run
"""
assert not Pkgbuild.ALLOW_MULTI_ARCHITECTURE_RUN
@@ -1,27 +0,0 @@
import argparse
from pytest_mock import MockerFixture
from ahriman.application.handlers.reload import Reload
from ahriman.core.configuration import Configuration
from ahriman.core.repository import Repository
def test_run(args: argparse.Namespace, configuration: Configuration, repository: Repository,
mocker: MockerFixture) -> None:
"""
must run command
"""
mocker.patch("ahriman.core.repository.Repository.load", return_value=repository)
run_mock = mocker.patch("ahriman.core.status.local_client.LocalClient.configuration_reload")
_, repository_id = configuration.check_loaded()
Reload.run(args, repository_id, configuration, report=False)
run_mock.assert_called_once_with()
def test_disallow_multi_architecture_run() -> None:
"""
must not allow multi architecture run
"""
assert not Reload.ALLOW_MULTI_ARCHITECTURE_RUN

Some files were not shown because too many files have changed in this diff Show More