Compare commits

...

25 Commits

Author SHA1 Message Date
2414686f3c review fixes 2022-04-17 17:59:39 +03:00
60bb880f9f make identation a bit more readable 2022-04-17 17:27:38 +03:00
3016a919c5 fix part of warnings for sphinx 2022-04-17 06:55:31 +03:00
e2f7e9cf28 add raises note
Also change behaviour of the `from_option` method to fallback to
disabled instead of raising exception on unknown option
2022-04-17 05:44:46 +03:00
5b6ba721fe migrate docstrings from reST to google format 2022-04-16 21:13:17 +03:00
0db619136d Release 2.0.0rc7 2022-04-11 00:48:08 +03:00
208a9b920d docs update 2022-04-11 00:46:46 +03:00
cb63bc08ff add backup and restore subcommands 2022-04-10 21:34:34 +03:00
6551c8d983 merge restore to rebuild commannd 2022-04-10 01:51:12 +03:00
a6c8d64053 Release 2.0.0rc6 2022-04-09 17:34:23 +03:00
fd78f2b5e2 do not render failed packages in jinja (#57)
basic templates require package info which is unavailable if package
wasn't built
2022-04-09 17:31:13 +03:00
900907cdaa Release 2.0.0rc5 2022-04-08 04:42:05 +03:00
5ff2f43506 change telegram default index to telegram-index 2022-04-08 04:32:34 +03:00
dd521b49b5 force git run from the same dir to clone 2022-04-08 04:04:06 +03:00
5b1f5a8473 fix users migration 2022-04-08 03:45:17 +03:00
86af13f09e add telegram integraion 2022-04-08 03:41:07 +03:00
733c014229 Release 2.0.0rc4 2022-04-08 01:14:35 +03:00
783c16b2ed trim versions before dependency list calculation
When dependencies list contains same package with version it tries to
find packages which don't exists
2022-04-07 20:32:55 +03:00
2536b8dc1f add support of repository restoration 2022-04-07 04:49:07 +03:00
e200ac9776 add support of officiall repositories api 2022-04-07 04:19:37 +03:00
6946745153 fix descriptions 2022-04-06 01:48:03 +03:00
6de75377c3 Release 2.0.0rc3 2022-04-04 02:40:17 +03:00
a734b86e66 allow any tag for push 2022-04-04 02:39:56 +03:00
74906d084a Release 2.0.0rc2 2022-04-04 02:27:19 +03:00
22d1d835af change regex for sed 2022-04-04 02:26:42 +03:00
210 changed files with 8142 additions and 4650 deletions

View File

@ -3,6 +3,9 @@ name: docker image
on: on:
push: push:
branches: [ master ] branches: [ master ]
tags:
- '*'
- '!*rc*'
jobs: jobs:
docker-image: docker-image:

View File

@ -3,7 +3,7 @@ name: release
on: on:
push: push:
tags: tags:
- '*.*.*' - '*'
jobs: jobs:
make-release: make-release:

View File

@ -24,7 +24,7 @@ archive_directory: $(TARGET_FILES)
find "$(PROJECT)" -depth -type d -name "*.egg-info" -execdir rm -rf {} + find "$(PROJECT)" -depth -type d -name "*.egg-info" -execdir rm -rf {} +
archlinux: archive archlinux: archive
sed -i "s/pkgver=[0-9.]*/pkgver=$(VERSION)/" package/archlinux/PKGBUILD sed -i "s/pkgver=.*/pkgver=$(VERSION)/" package/archlinux/PKGBUILD
check: clean check: clean
tox -e check tox -e check
@ -53,4 +53,4 @@ version:
ifndef VERSION ifndef VERSION
$(error VERSION is required, but not set) $(error VERSION is required, but not set)
endif endif
sed -i '/__version__ = "[0-9.]*/s/[^"][^)]*/__version__ = "$(VERSION)"/' src/ahriman/version.py sed -i '/__version__ = .*/s/[^"][^)]*/__version__ = "$(VERSION)"/' src/ahriman/version.py

View File

@ -13,7 +13,7 @@ Wrapper for managing custom repository inspired by [repo-scripts](https://github
* Multi-architecture support. * Multi-architecture support.
* VCS packages support. * VCS packages support.
* Sign support with gpg (repository, package, per package settings). * Sign support with gpg (repository, package, per package settings).
* Synchronization to remote services (rsync, s3 and github) and report generation (html). * Synchronization to remote services (rsync, s3 and github) and report generation (email, html, telegram).
* Dependency manager. * Dependency manager.
* Ability to patch AUR packages and even create package from local PKGBUILDs. * Ability to patch AUR packages and even create package from local PKGBUILDs.
* Repository status interface with optional authorization and control options: * Repository status interface with optional authorization and control options:

File diff suppressed because it is too large Load Diff

Before

Width:  |  Height:  |  Size: 503 KiB

After

Width:  |  Height:  |  Size: 540 KiB

View File

@ -3,7 +3,7 @@
ahriman ahriman
.SH SYNOPSIS .SH SYNOPSIS
.B ahriman .B ahriman
[-h] [-a ARCHITECTURE] [-c CONFIGURATION] [--force] [-l LOCK] [--no-report] [-q] [--unsafe] [-v] {aur-search,search,help,help-commands-unsafe,key-import,package-add,add,package-update,package-remove,remove,package-status,status,package-status-remove,package-status-update,status-update,patch-add,patch-list,patch-remove,repo-check,check,repo-clean,clean,repo-config,config,repo-rebuild,rebuild,repo-remove-unknown,remove-unknown,repo-report,report,repo-setup,init,repo-init,setup,repo-sign,sign,repo-status-update,repo-sync,sync,repo-update,update,user-add,user-list,user-remove,web} ... [-h] [-a ARCHITECTURE] [-c CONFIGURATION] [--force] [-l LOCK] [--no-report] [-q] [--unsafe] [-v] {aur-search,search,help,help-commands-unsafe,key-import,package-add,add,package-update,package-remove,remove,package-status,status,package-status-remove,package-status-update,status-update,patch-add,patch-list,patch-remove,repo-backup,repo-check,check,repo-clean,clean,repo-config,config,repo-rebuild,rebuild,repo-remove-unknown,remove-unknown,repo-report,report,repo-restore,repo-setup,init,repo-init,setup,repo-sign,sign,repo-status-update,repo-sync,sync,repo-update,update,user-add,user-list,user-remove,web} ...
.SH DESCRIPTION .SH DESCRIPTION
ArcH Linux ReposItory MANager ArcH Linux ReposItory MANager
.SH OPTIONS .SH OPTIONS
@ -79,6 +79,9 @@ list patch sets
\fBahriman\fR \fI\,patch-remove\/\fR \fBahriman\fR \fI\,patch-remove\/\fR
remove patch set remove patch set
.TP .TP
\fBahriman\fR \fI\,repo-backup\/\fR
backup repository data
.TP
\fBahriman\fR \fI\,repo-check\/\fR \fBahriman\fR \fI\,repo-check\/\fR
check for updates check for updates
.TP .TP
@ -97,6 +100,9 @@ remove unknown packages
\fBahriman\fR \fI\,repo-report\/\fR \fBahriman\fR \fI\,repo-report\/\fR
generate report generate report
.TP .TP
\fBahriman\fR \fI\,repo-restore\/\fR
restore repository data
.TP
\fBahriman\fR \fI\,repo-setup\/\fR \fBahriman\fR \fI\,repo-setup\/\fR
initial service configuration initial service configuration
.TP .TP
@ -207,7 +213,7 @@ key server for key import
.SH OPTIONS 'ahriman package-add' .SH OPTIONS 'ahriman package-add'
usage: ahriman package-add [-h] [-e] [-n] usage: ahriman package-add [-h] [-e] [-n]
[-s {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote}] [-s {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote,PackageSource.Repository}]
[--without-dependencies] [--without-dependencies]
package [package ...] package [package ...]
@ -226,7 +232,7 @@ return non\-zero exit status if result is empty
run update function after run update function after
.TP .TP
\fB\-s\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote}, \fB\-\-source\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote} \fB\-s\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote,PackageSource.Repository}, \fB\-\-source\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote,PackageSource.Repository}
explicitly specify the package source for this command explicitly specify the package source for this command
.TP .TP
@ -235,7 +241,7 @@ do not add dependencies
.SH OPTIONS 'ahriman add' .SH OPTIONS 'ahriman add'
usage: ahriman package-add [-h] [-e] [-n] usage: ahriman package-add [-h] [-e] [-n]
[-s {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote}] [-s {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote,PackageSource.Repository}]
[--without-dependencies] [--without-dependencies]
package [package ...] package [package ...]
@ -254,7 +260,7 @@ return non\-zero exit status if result is empty
run update function after run update function after
.TP .TP
\fB\-s\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote}, \fB\-\-source\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote} \fB\-s\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote,PackageSource.Repository}, \fB\-\-source\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote,PackageSource.Repository}
explicitly specify the package source for this command explicitly specify the package source for this command
.TP .TP
@ -263,7 +269,7 @@ do not add dependencies
.SH OPTIONS 'ahriman package-update' .SH OPTIONS 'ahriman package-update'
usage: ahriman package-add [-h] [-e] [-n] usage: ahriman package-add [-h] [-e] [-n]
[-s {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote}] [-s {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote,PackageSource.Repository}]
[--without-dependencies] [--without-dependencies]
package [package ...] package [package ...]
@ -282,7 +288,7 @@ return non\-zero exit status if result is empty
run update function after run update function after
.TP .TP
\fB\-s\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote}, \fB\-\-source\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote} \fB\-s\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote,PackageSource.Repository}, \fB\-\-source\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local,PackageSource.Remote,PackageSource.Repository}
explicitly specify the package source for this command explicitly specify the package source for this command
.TP .TP
@ -439,6 +445,16 @@ remove patches for the package
package base package base
.SH OPTIONS 'ahriman repo-backup'
usage: ahriman repo-backup [-h] path
backup settings and database
.TP
\fBpath\fR
path of the output archive
.SH OPTIONS 'ahriman repo-check' .SH OPTIONS 'ahriman repo-check'
usage: ahriman repo-check [-h] [-e] [--no-vcs] [package ...] usage: ahriman repo-check [-h] [-e] [--no-vcs] [package ...]
@ -532,7 +548,7 @@ dump configuration for the specified architecture
.SH OPTIONS 'ahriman repo-rebuild' .SH OPTIONS 'ahriman repo-rebuild'
usage: ahriman repo-rebuild [-h] [--depends-on DEPENDS_ON] [--dry-run] [-e] usage: ahriman repo-rebuild [-h] [--depends-on DEPENDS_ON] [--dry-run] [--from-database] [-e]
force rebuild whole repository force rebuild whole repository
@ -545,12 +561,18 @@ only rebuild packages that depend on specified package
\fB\-\-dry\-run\fR \fB\-\-dry\-run\fR
just perform check for packages without rebuild process itself just perform check for packages without rebuild process itself
.TP
\fB\-\-from\-database\fR
read packages from database instead of filesystem. This feature in particular is required in case if you would like to
restore repository from another repository instance. Note however that in order to restore packages you need to have
original ahriman instance run with web service and have run repo\-update at least once.
.TP .TP
\fB\-e\fR, \fB\-\-exit\-code\fR \fB\-e\fR, \fB\-\-exit\-code\fR
return non\-zero exit status if result is empty return non\-zero exit status if result is empty
.SH OPTIONS 'ahriman rebuild' .SH OPTIONS 'ahriman rebuild'
usage: ahriman repo-rebuild [-h] [--depends-on DEPENDS_ON] [--dry-run] [-e] usage: ahriman repo-rebuild [-h] [--depends-on DEPENDS_ON] [--dry-run] [--from-database] [-e]
force rebuild whole repository force rebuild whole repository
@ -563,6 +585,12 @@ only rebuild packages that depend on specified package
\fB\-\-dry\-run\fR \fB\-\-dry\-run\fR
just perform check for packages without rebuild process itself just perform check for packages without rebuild process itself
.TP
\fB\-\-from\-database\fR
read packages from database instead of filesystem. This feature in particular is required in case if you would like to
restore repository from another repository instance. Note however that in order to restore packages you need to have
original ahriman instance run with web service and have run repo\-update at least once.
.TP .TP
\fB\-e\fR, \fB\-\-exit\-code\fR \fB\-e\fR, \fB\-\-exit\-code\fR
return non\-zero exit status if result is empty return non\-zero exit status if result is empty
@ -615,6 +643,19 @@ generate repository report according to current settings
target to generate report target to generate report
.SH OPTIONS 'ahriman repo-restore'
usage: ahriman repo-restore [-h] [-o OUTPUT] path
restore settings and database
.TP
\fBpath\fR
path of the input archive
.TP
\fB\-o\fR \fI\,OUTPUT\/\fR, \fB\-\-output\fR \fI\,OUTPUT\/\fR
root path of the extracted files
.SH OPTIONS 'ahriman repo-setup' .SH OPTIONS 'ahriman repo-setup'
usage: ahriman repo-setup [-h] [--build-as-user BUILD_AS_USER] [--build-command BUILD_COMMAND] usage: ahriman repo-setup [-h] [--build-as-user BUILD_AS_USER] [--build-command BUILD_COMMAND]
[--from-configuration FROM_CONFIGURATION] [--no-multilib] --packager PACKAGER --repository [--from-configuration FROM_CONFIGURATION] [--no-multilib] --packager PACKAGER --repository

View File

@ -114,6 +114,18 @@ Section name must be either `html` (plus optional architecture name, e.g. `html:
* `link_path` - prefix for HTML links, string, required. * `link_path` - prefix for HTML links, string, required.
* `template_path` - path to Jinja2 template, string, required. * `template_path` - path to Jinja2 template, string, required.
### `telegram` type
Section name must be either `telegram` (plus optional architecture name, e.g. `telegram:x86_64`) or random name with `type` set.
* `type` - type of the report, string, optional, must be set to `telegram` if exists.
* `api_key` - telegram bot API key, string, required. Please refer FAQ about how to create chat and bot
* `chat_id` - telegram chat id, either string with `@` or integer value, required.
* `homepage` - link to homepage, string, optional.
* `link_path` - prefix for HTML links, string, required.
* `template_path` - path to Jinja2 template, string, required.
* `template_type` - `parse_mode` to be passed to telegram API, one of `MarkdownV2`, `HTML`, `Markdown`, string, optional, default `HTML`.
## `upload` group ## `upload` group
Remote synchronization settings. Remote synchronization settings.

View File

@ -199,7 +199,7 @@ server {
## Docker image ## Docker image
We provide official images which can be found under `arcan1s/ahriman` repository. Docker image is being updated on each master commit as well as on each version. If you would like to use last (probably unstable build) you can use `latest` tag; otherwise you can use any version tag available. We provide official images which can be found under `arcan1s/ahriman` repository. Docker image is being updated on each master commit as well as on each version. If you would like to use last (probably unstable) build you can use `edge` tag or `latest` for any tagged versions; otherwise you can use any version tag available.
The default action (in case if no arguments provided) is `repo-update`. Basically the idea is to run container, e.g.: The default action (in case if no arguments provided) is `repo-update`. Basically the idea is to run container, e.g.:
@ -242,7 +242,7 @@ You can pass any of these variables by using `-e` argument, e.g.:
docker run -e AHRIMAN_PORT=8080 arcan1s/ahriman:latest docker run -e AHRIMAN_PORT=8080 arcan1s/ahriman:latest
``` ```
### Working with web service ### Web service setup
Well for that you would need to have web container instance running forever; it can be achieved by the following command: Well for that you would need to have web container instance running forever; it can be achieved by the following command:
@ -402,6 +402,46 @@ There are several choices:
After these steps `index.html` file will be automatically synced to S3 After these steps `index.html` file will be automatically synced to S3
### I would like to get messages to my telegram account/channel
1. It still requires additional dependencies:
```shell
yay -S python-jinja
```
2. Register bot in telegram. You can do it by talking with [@BotFather](https://t.me/botfather). For more details please refer to [official documentation](https://core.telegram.org/bots).
3. Optionally (if you want to post message in chat):
1. Create telegram channel.
2. Invite your bot into the channel.
3. Make your channel public
4. Get chat id if you want to use by numerical id or just use id prefixed with `@` (e.g. `@ahriman`). If you are not using chat the chat id is your user id. If you don't want to make channel public you can use [this guide](https://stackoverflow.com/a/33862907).
5. Configure the service:
```ini
[report]
target = telegram
[telegram]
api_key = aaAAbbBBccCC
chat_id = @ahriman
link_path = http://example.com/x86_64
```
`api_key` is the one sent by [@BotFather](https://t.me/botfather), `chat_id` is the value retrieved from previous step.
If you did everything fine you should receive the message with the next update. Quick credentials check can be done by using the following command:
```shell
curl 'https://api.telegram.org/bot${CHAT_ID}/sendMessage?chat_id=${API_KEY}&text=hello'
```
(replace `${CHAT_ID}` and `${API_KEY}` with the values from configuration).
## Web service ## Web service
### Readme mentions web interface, how do I use it? ### Readme mentions web interface, how do I use it?
@ -479,6 +519,36 @@ After these steps `index.html` file will be automatically synced to S3
5. Create end-user `sudo -u ahriman ahriman user-add -r write my-first-user`. When it will ask for the password leave it blank. 5. Create end-user `sudo -u ahriman ahriman user-add -r write my-first-user`. When it will ask for the password leave it blank.
6. Restart web service `systemctl restart ahriman-web@x86_64`. 6. Restart web service `systemctl restart ahriman-web@x86_64`.
## Backup and restore
The service provides several commands aim to do easy repository backup and restore. If you would like to move repository from the server `server1.example.com` to another `server2.example.com` you have to perform the following steps:
1. On the source server `server1.example.com` run `repo-backup` command, e.g.:
```shell
sudo ahriman repo-backup /tmp/repo.tar.gz
```
This command will pack all configuration files together with database file into the archive specified as command line argument (i.e. `/tmp/repo.tar.gz`). In addition it will also archive `cache` directory (the one which contains local clones used by e.g. local packages) and `.gnupg` of the `ahriman` user.
2. Copy created archive from source server `server1.example.com` to target `server2.example.com`.
3. Install ahriman as usual on the target server `server2.example.com` if you didn't yet.
4. Extract archive e.g. by using subcommand:
```shell
sudo ahriman repo-restore /tmp/repo.tar.gz
```
An additional argument `-o`/`--output` can be used to specify extraction root (`/` by default).
5. Rebuild repository:
```shell
sudo -u ahriman ahriman repo-rebuild --from-database
```
## Other topics ## Other topics
### How does it differ from %another-manager%? ### How does it differ from %another-manager%?
@ -518,6 +588,10 @@ Though originally I've created ahriman by trying to improve the project, it stil
`repo-scripts` also have bad architecture and bad quality code and uses out-of-dated `yaourt` and `package-query`. `repo-scripts` also have bad architecture and bad quality code and uses out-of-dated `yaourt` and `package-query`.
#### [toolbox](https://github.com/chaotic-aur/toolbox)
It is automation tools for `repoctl` mentioned above. Except for using shell it looks pretty cool and also offers some additional features like patches, remote synchronization (isn't it?) and reporting.
### I would like to check service logs ### I would like to check service logs
By default, the service writes logs to `/dev/log` which can be accessed by using `journalctl` command (logs are written to the journal of the user under which command is run). By default, the service writes logs to `/dev/log` which can be accessed by using `journalctl` command (logs are written to the journal of the user under which command is run).
@ -528,7 +602,7 @@ You can also edit configuration and forward logs to `stderr`, just change `handl
sed -i 's/handlers = syslog_handler/handlers = console_handler/g' /etc/ahriman.ini.d/logging.ini sed -i 's/handlers = syslog_handler/handlers = console_handler/g' /etc/ahriman.ini.d/logging.ini
``` ```
You can even configure logging as you wish, but kindly refer to python `logging` module configuration. You can even configure logging as you wish, but kindly refer to python `logging` module [configuration](https://docs.python.org/3/library/logging.config.html).
### Html customization ### Html customization

View File

@ -1,7 +1,7 @@
# Maintainer: Evgeniy Alekseev # Maintainer: Evgeniy Alekseev
pkgname='ahriman' pkgname='ahriman'
pkgver=2.0.0-rc1 pkgver=2.0.0rc7
pkgrel=1 pkgrel=1
pkgdesc="ArcH Linux ReposItory MANager" pkgdesc="ArcH Linux ReposItory MANager"
arch=('any') arch=('any')
@ -38,7 +38,7 @@ build() {
package() { package() {
cd "$pkgname" cd "$pkgname"
python -m installer --destdir="$pkgdir" dist/*.whl python -m installer --destdir="$pkgdir" "dist/$pkgname-$pkgver-py3-none-any.whl"
# python-installer actually thinks that you cannot just copy files to root # python-installer actually thinks that you cannot just copy files to root
# thus we need to copy them manually # thus we need to copy them manually

View File

@ -45,6 +45,9 @@ ssl = disabled
[html] [html]
template_path = /usr/share/ahriman/templates/repo-index.jinja2 template_path = /usr/share/ahriman/templates/repo-index.jinja2
[telegram]
template_path = /usr/share/ahriman/templates/telegram-index.jinja2
[upload] [upload]
target = target =

View File

@ -0,0 +1,4 @@
{#simplified version of full report#}
<b>{{ repository }} update</b>
{% for package in packages %}
<a href="{{ link_path }}/{{ package.filename }}">{{ package.name }}</a> {{ package.version }}{% endfor %}

View File

@ -67,6 +67,7 @@ setup(
"package/share/ahriman/templates/build-status.jinja2", "package/share/ahriman/templates/build-status.jinja2",
"package/share/ahriman/templates/email-index.jinja2", "package/share/ahriman/templates/email-index.jinja2",
"package/share/ahriman/templates/repo-index.jinja2", "package/share/ahriman/templates/repo-index.jinja2",
"package/share/ahriman/templates/telegram-index.jinja2",
]), ]),
("share/ahriman/templates/build-status", [ ("share/ahriman/templates/build-status", [
"package/share/ahriman/templates/build-status/login-modal.jinja2", "package/share/ahriman/templates/build-status/login-modal.jinja2",

View File

@ -43,8 +43,12 @@ SubParserAction = TypeVar("SubParserAction", bound="argparse._SubParsersAction[a
def _formatter(prog: str) -> argparse.HelpFormatter: def _formatter(prog: str) -> argparse.HelpFormatter:
""" """
formatter for the help message formatter for the help message
:param prog: application name
:return: formatter used by default Args:
prog(str): application name
Returns:
argparse.HelpFormatter: formatter used by default
""" """
return argparse.ArgumentDefaultsHelpFormatter(prog, width=120) return argparse.ArgumentDefaultsHelpFormatter(prog, width=120)
@ -52,7 +56,9 @@ def _formatter(prog: str) -> argparse.HelpFormatter:
def _parser() -> argparse.ArgumentParser: def _parser() -> argparse.ArgumentParser:
""" """
command line parser generator command line parser generator
:return: command line parser for the application
Returns:
argparse.ArgumentParser: command line parser for the application
""" """
parser = argparse.ArgumentParser(prog="ahriman", description="ArcH Linux ReposItory MANager", parser = argparse.ArgumentParser(prog="ahriman", description="ArcH Linux ReposItory MANager",
epilog="Argument list can also be read from file by using @ prefix.", epilog="Argument list can also be read from file by using @ prefix.",
@ -83,12 +89,14 @@ def _parser() -> argparse.ArgumentParser:
_set_patch_add_parser(subparsers) _set_patch_add_parser(subparsers)
_set_patch_list_parser(subparsers) _set_patch_list_parser(subparsers)
_set_patch_remove_parser(subparsers) _set_patch_remove_parser(subparsers)
_set_repo_backup_parser(subparsers)
_set_repo_check_parser(subparsers) _set_repo_check_parser(subparsers)
_set_repo_clean_parser(subparsers) _set_repo_clean_parser(subparsers)
_set_repo_config_parser(subparsers) _set_repo_config_parser(subparsers)
_set_repo_rebuild_parser(subparsers) _set_repo_rebuild_parser(subparsers)
_set_repo_remove_unknown_parser(subparsers) _set_repo_remove_unknown_parser(subparsers)
_set_repo_report_parser(subparsers) _set_repo_report_parser(subparsers)
_set_repo_restore_parser(subparsers)
_set_repo_setup_parser(subparsers) _set_repo_setup_parser(subparsers)
_set_repo_sign_parser(subparsers) _set_repo_sign_parser(subparsers)
_set_repo_status_update_parser(subparsers) _set_repo_status_update_parser(subparsers)
@ -105,8 +113,12 @@ def _parser() -> argparse.ArgumentParser:
def _set_aur_search_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_aur_search_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for AUR search subcommand add parser for AUR search subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("aur-search", aliases=["search"], help="search for package", parser = root.add_parser("aur-search", aliases=["search"], help="search for package",
description="search for package in AUR using API", formatter_class=_formatter) description="search for package in AUR using API", formatter_class=_formatter)
@ -124,8 +136,12 @@ def _set_aur_search_parser(root: SubParserAction) -> argparse.ArgumentParser:
def _set_help_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_help_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for listing help subcommand add parser for listing help subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("help", help="show help message", parser = root.add_parser("help", help="show help message",
description="show help message for application or command and exit", description="show help message for application or command and exit",
@ -139,8 +155,12 @@ def _set_help_parser(root: SubParserAction) -> argparse.ArgumentParser:
def _set_help_commands_unsafe_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_help_commands_unsafe_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for listing unsafe commands add parser for listing unsafe commands
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("help-commands-unsafe", help="list unsafe commands", parser = root.add_parser("help-commands-unsafe", help="list unsafe commands",
description="list unsafe commands as defined in default args", formatter_class=_formatter) description="list unsafe commands as defined in default args", formatter_class=_formatter)
@ -154,8 +174,12 @@ def _set_help_commands_unsafe_parser(root: SubParserAction) -> argparse.Argument
def _set_key_import_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_key_import_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for key import subcommand add parser for key import subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("key-import", help="import PGP key", parser = root.add_parser("key-import", help="import PGP key",
description="import PGP key from public sources to the repository user", description="import PGP key from public sources to the repository user",
@ -173,8 +197,12 @@ def _set_key_import_parser(root: SubParserAction) -> argparse.ArgumentParser:
def _set_package_add_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_package_add_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for package addition subcommand add parser for package addition subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("package-add", aliases=["add", "package-update"], help="add package", parser = root.add_parser("package-add", aliases=["add", "package-update"], help="add package",
description="add existing or new package to the build queue", description="add existing or new package to the build queue",
@ -202,8 +230,12 @@ def _set_package_add_parser(root: SubParserAction) -> argparse.ArgumentParser:
def _set_package_remove_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_package_remove_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for package removal subcommand add parser for package removal subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("package-remove", aliases=["remove"], help="remove package", parser = root.add_parser("package-remove", aliases=["remove"], help="remove package",
description="remove package from the repository", formatter_class=_formatter) description="remove package from the repository", formatter_class=_formatter)
@ -215,8 +247,12 @@ def _set_package_remove_parser(root: SubParserAction) -> argparse.ArgumentParser
def _set_package_status_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_package_status_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for package status subcommand add parser for package status subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("package-status", aliases=["status"], help="get package status", parser = root.add_parser("package-status", aliases=["status"], help="get package status",
description="request status of the package", description="request status of the package",
@ -235,8 +271,12 @@ def _set_package_status_parser(root: SubParserAction) -> argparse.ArgumentParser
def _set_package_status_remove_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_package_status_remove_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for package status remove subcommand add parser for package status remove subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("package-status-remove", help="remove package status", parser = root.add_parser("package-status-remove", help="remove package status",
description="remove the package from the status page", description="remove the package from the status page",
@ -252,8 +292,12 @@ def _set_package_status_remove_parser(root: SubParserAction) -> argparse.Argumen
def _set_package_status_update_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_package_status_update_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for package status update subcommand add parser for package status update subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("package-status-update", aliases=["status-update"], help="update package status", parser = root.add_parser("package-status-update", aliases=["status-update"], help="update package status",
description="update package status on the status page", formatter_class=_formatter) description="update package status on the status page", formatter_class=_formatter)
@ -270,8 +314,12 @@ def _set_package_status_update_parser(root: SubParserAction) -> argparse.Argumen
def _set_patch_add_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_patch_add_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for new patch subcommand add parser for new patch subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("patch-add", help="add patch set", description="create or update source patches", parser = root.add_parser("patch-add", help="add patch set", description="create or update source patches",
epilog="In order to add a patch set for the package you will need to clone " epilog="In order to add a patch set for the package you will need to clone "
@ -290,8 +338,12 @@ def _set_patch_add_parser(root: SubParserAction) -> argparse.ArgumentParser:
def _set_patch_list_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_patch_list_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for list patches subcommand add parser for list patches subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("patch-list", help="list patch sets", parser = root.add_parser("patch-list", help="list patch sets",
description="list available patches for the package", formatter_class=_formatter) description="list available patches for the package", formatter_class=_formatter)
@ -304,8 +356,12 @@ def _set_patch_list_parser(root: SubParserAction) -> argparse.ArgumentParser:
def _set_patch_remove_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_patch_remove_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for remove patches subcommand add parser for remove patches subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("patch-remove", help="remove patch set", description="remove patches for the package", parser = root.add_parser("patch-remove", help="remove patch set", description="remove patches for the package",
formatter_class=_formatter) formatter_class=_formatter)
@ -314,11 +370,32 @@ def _set_patch_remove_parser(root: SubParserAction) -> argparse.ArgumentParser:
return parser return parser
def _set_repo_backup_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for repository backup subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("repo-backup", help="backup repository data",
description="backup settings and database", formatter_class=_formatter)
parser.add_argument("path", help="path of the output archive", type=Path)
parser.set_defaults(handler=handlers.Backup, architecture=[""], lock=None, no_report=True, unsafe=True)
return parser
def _set_repo_check_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_repo_check_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for repository check subcommand add parser for repository check subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("repo-check", aliases=["check"], help="check for updates", parser = root.add_parser("repo-check", aliases=["check"], help="check for updates",
description="check for packages updates. Same as update --dry-run --no-manual", description="check for packages updates. Same as update --dry-run --no-manual",
@ -333,8 +410,12 @@ def _set_repo_check_parser(root: SubParserAction) -> argparse.ArgumentParser:
def _set_repo_clean_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_repo_clean_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for repository clean subcommand add parser for repository clean subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("repo-clean", aliases=["clean"], help="clean local caches", parser = root.add_parser("repo-clean", aliases=["clean"], help="clean local caches",
description="remove local caches", description="remove local caches",
@ -353,8 +434,12 @@ def _set_repo_clean_parser(root: SubParserAction) -> argparse.ArgumentParser:
def _set_repo_config_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_repo_config_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for config subcommand add parser for config subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("repo-config", aliases=["config"], help="dump configuration", parser = root.add_parser("repo-config", aliases=["config"], help="dump configuration",
description="dump configuration for the specified architecture", description="dump configuration for the specified architecture",
@ -366,14 +451,24 @@ def _set_repo_config_parser(root: SubParserAction) -> argparse.ArgumentParser:
def _set_repo_rebuild_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_repo_rebuild_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for repository rebuild subcommand add parser for repository rebuild subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("repo-rebuild", aliases=["rebuild"], help="rebuild repository", parser = root.add_parser("repo-rebuild", aliases=["rebuild"], help="rebuild repository",
description="force rebuild whole repository", formatter_class=_formatter) description="force rebuild whole repository", formatter_class=_formatter)
parser.add_argument("--depends-on", help="only rebuild packages that depend on specified package", action="append") parser.add_argument("--depends-on", help="only rebuild packages that depend on specified package", action="append")
parser.add_argument("--dry-run", help="just perform check for packages without rebuild process itself", parser.add_argument("--dry-run", help="just perform check for packages without rebuild process itself",
action="store_true") action="store_true")
parser.add_argument("--from-database",
help="read packages from database instead of filesystem. This feature in particular is "
"required in case if you would like to restore repository from another repository "
"instance. Note however that in order to restore packages you need to have original "
"ahriman instance run with web service and have run repo-update at least once.",
action="store_true")
parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty", action="store_true") parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty", action="store_true")
parser.set_defaults(handler=handlers.Rebuild) parser.set_defaults(handler=handlers.Rebuild)
return parser return parser
@ -382,8 +477,12 @@ def _set_repo_rebuild_parser(root: SubParserAction) -> argparse.ArgumentParser:
def _set_repo_remove_unknown_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_repo_remove_unknown_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for remove unknown packages subcommand add parser for remove unknown packages subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("repo-remove-unknown", aliases=["remove-unknown"], help="remove unknown packages", parser = root.add_parser("repo-remove-unknown", aliases=["remove-unknown"], help="remove unknown packages",
description="remove packages which are missing in AUR and do not have local PKGBUILDs", description="remove packages which are missing in AUR and do not have local PKGBUILDs",
@ -397,8 +496,12 @@ def _set_repo_remove_unknown_parser(root: SubParserAction) -> argparse.ArgumentP
def _set_repo_report_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_repo_report_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for report subcommand add parser for report subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("repo-report", aliases=["report"], help="generate report", parser = root.add_parser("repo-report", aliases=["report"], help="generate report",
description="generate repository report according to current settings", description="generate repository report according to current settings",
@ -409,11 +512,33 @@ def _set_repo_report_parser(root: SubParserAction) -> argparse.ArgumentParser:
return parser return parser
def _set_repo_restore_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for repository restore subcommand
Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("repo-restore", help="restore repository data",
description="restore settings and database", formatter_class=_formatter)
parser.add_argument("path", help="path of the input archive", type=Path)
parser.add_argument("-o", "--output", help="root path of the extracted files", type=Path, default=Path("/"))
parser.set_defaults(handler=handlers.Restore, architecture=[""], lock=None, no_report=True, unsafe=True)
return parser
def _set_repo_setup_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_repo_setup_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for setup subcommand add parser for setup subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("repo-setup", aliases=["init", "repo-init", "setup"], help="initial service configuration", parser = root.add_parser("repo-setup", aliases=["init", "repo-init", "setup"], help="initial service configuration",
description="create initial service configuration, requires root", description="create initial service configuration, requires root",
@ -437,8 +562,12 @@ def _set_repo_setup_parser(root: SubParserAction) -> argparse.ArgumentParser:
def _set_repo_sign_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_repo_sign_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for sign subcommand add parser for sign subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("repo-sign", aliases=["sign"], help="sign packages", parser = root.add_parser("repo-sign", aliases=["sign"], help="sign packages",
description="(re-)sign packages and repository database according to current settings", description="(re-)sign packages and repository database according to current settings",
@ -452,8 +581,12 @@ def _set_repo_sign_parser(root: SubParserAction) -> argparse.ArgumentParser:
def _set_repo_status_update_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_repo_status_update_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for repository status update subcommand add parser for repository status update subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("repo-status-update", help="update repository status", parser = root.add_parser("repo-status-update", help="update repository status",
description="update repository status on the status page", formatter_class=_formatter) description="update repository status on the status page", formatter_class=_formatter)
@ -467,8 +600,12 @@ def _set_repo_status_update_parser(root: SubParserAction) -> argparse.ArgumentPa
def _set_repo_sync_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_repo_sync_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for repository sync subcommand add parser for repository sync subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("repo-sync", aliases=["sync"], help="sync repository", parser = root.add_parser("repo-sync", aliases=["sync"], help="sync repository",
description="sync repository files to remote server according to current settings", description="sync repository files to remote server according to current settings",
@ -482,8 +619,12 @@ def _set_repo_sync_parser(root: SubParserAction) -> argparse.ArgumentParser:
def _set_repo_update_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_repo_update_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for repository update subcommand add parser for repository update subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("repo-update", aliases=["update"], help="update packages", parser = root.add_parser("repo-update", aliases=["update"], help="update packages",
description="check for packages updates and run build process if requested", description="check for packages updates and run build process if requested",
@ -502,8 +643,12 @@ def _set_repo_update_parser(root: SubParserAction) -> argparse.ArgumentParser:
def _set_user_add_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_user_add_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for create user subcommand add parser for create user subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("user-add", help="create or update user", parser = root.add_parser("user-add", help="create or update user",
description="update user for web services with the given password and role. " description="update user for web services with the given password and role. "
@ -524,8 +669,12 @@ def _set_user_add_parser(root: SubParserAction) -> argparse.ArgumentParser:
def _set_user_list_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_user_list_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for user list subcommand add parser for user list subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("user-list", help="user known users and their access", parser = root.add_parser("user-list", help="user known users and their access",
description="list users from the user mapping and their roles", description="list users from the user mapping and their roles",
@ -541,8 +690,12 @@ def _set_user_list_parser(root: SubParserAction) -> argparse.ArgumentParser:
def _set_user_remove_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_user_remove_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for user removal subcommand add parser for user removal subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("user-remove", help="remove user", parser = root.add_parser("user-remove", help="remove user",
description="remove user from the user mapping and update the configuration", description="remove user from the user mapping and update the configuration",
@ -557,8 +710,12 @@ def _set_user_remove_parser(root: SubParserAction) -> argparse.ArgumentParser:
def _set_web_parser(root: SubParserAction) -> argparse.ArgumentParser: def _set_web_parser(root: SubParserAction) -> argparse.ArgumentParser:
""" """
add parser for web subcommand add parser for web subcommand
:param root: subparsers for the commands
:return: created argument parser Args:
root(SubParserAction): subparsers for the commands
Returns:
argparse.ArgumentParser: created argument parser
""" """
parser = root.add_parser("web", help="web server", description="start web server", formatter_class=_formatter) parser = root.add_parser("web", help="web server", description="start web server", formatter_class=_formatter)
parser.set_defaults(handler=handlers.Web, lock=None, no_report=True, parser=_parser) parser.set_defaults(handler=handlers.Web, lock=None, no_report=True, parser=_parser)

View File

@ -32,7 +32,9 @@ class Application(Packages, Repository):
def _finalize(self, result: Result) -> None: def _finalize(self, result: Result) -> None:
""" """
generate report and sync to remote server generate report and sync to remote server
:param result: build result
Args:
result(Result): build result
""" """
self.report([], result) self.report([], result)
self.sync([], result.success) self.sync([], result.success)
@ -40,7 +42,9 @@ class Application(Packages, Repository):
def _known_packages(self) -> Set[str]: def _known_packages(self) -> Set[str]:
""" """
load packages from repository and pacman repositories load packages from repository and pacman repositories
:return: list of known packages
Returns:
Set[str]: list of known packages
""" """
known_packages: Set[str] = set() known_packages: Set[str] = set()
# local set # local set

View File

@ -39,21 +39,33 @@ class Packages(Properties):
def _finalize(self, result: Result) -> None: def _finalize(self, result: Result) -> None:
""" """
generate report and sync to remote server generate report and sync to remote server
:param result: build result
Args:
result(Result): build result
Raises:
NotImplementedError: not implemented method
""" """
raise NotImplementedError raise NotImplementedError
def _known_packages(self) -> Set[str]: def _known_packages(self) -> Set[str]:
""" """
load packages from repository and pacman repositories load packages from repository and pacman repositories
:return: list of known packages
Returns:
Set[str]: list of known packages
Raises:
NotImplementedError: not implemented method
""" """
raise NotImplementedError raise NotImplementedError
def _add_archive(self, source: str, *_: Any) -> None: def _add_archive(self, source: str, *_: Any) -> None:
""" """
add package from archive add package from archive
:param source: path to package archive
Args:
source(str): path to package archive
""" """
local_path = Path(source) local_path = Path(source)
dst = self.repository.paths.packages / local_path.name dst = self.repository.paths.packages / local_path.name
@ -62,12 +74,15 @@ class Packages(Properties):
def _add_aur(self, source: str, known_packages: Set[str], without_dependencies: bool) -> None: def _add_aur(self, source: str, known_packages: Set[str], without_dependencies: bool) -> None:
""" """
add package from AUR add package from AUR
:param source: package base name
:param known_packages: list of packages which are known by the service Args:
:param without_dependencies: if set, dependency check will be disabled source(str): package base name
known_packages(Set[str]): list of packages which are known by the service
without_dependencies(bool): if set, dependency check will be disabled
""" """
package = Package.load(source, PackageSource.AUR, self.repository.pacman, self.repository.aur_url) package = Package.load(source, PackageSource.AUR, self.repository.pacman, self.repository.aur_url)
self.repository.database.build_queue_insert(package)
self.database.build_queue_insert(package)
with tmpdir() as local_path: with tmpdir() as local_path:
Sources.load(local_path, package.git_url, self.database.patches_get(package.base)) Sources.load(local_path, package.git_url, self.database.patches_get(package.base))
@ -76,7 +91,9 @@ class Packages(Properties):
def _add_directory(self, source: str, *_: Any) -> None: def _add_directory(self, source: str, *_: Any) -> None:
""" """
add packages from directory add packages from directory
:param source: path to local directory
Args:
source(str): path to local directory
""" """
local_path = Path(source) local_path = Path(source)
for full_path in filter(package_like, local_path.iterdir()): for full_path in filter(package_like, local_path.iterdir()):
@ -85,22 +102,27 @@ class Packages(Properties):
def _add_local(self, source: str, known_packages: Set[str], without_dependencies: bool) -> None: def _add_local(self, source: str, known_packages: Set[str], without_dependencies: bool) -> None:
""" """
add package from local PKGBUILDs add package from local PKGBUILDs
:param source: path to directory with local source files
:param known_packages: list of packages which are known by the service Args:
:param without_dependencies: if set, dependency check will be disabled source(str): path to directory with local source files
known_packages(Set[str]): list of packages which are known by the service
without_dependencies(bool): if set, dependency check will be disabled
""" """
package = Package.load(source, PackageSource.Local, self.repository.pacman, self.repository.aur_url) package = Package.load(source, PackageSource.Local, self.repository.pacman, self.repository.aur_url)
cache_dir = self.repository.paths.cache_for(package.base) cache_dir = self.repository.paths.cache_for(package.base)
shutil.copytree(Path(source), cache_dir) # copy package to store in caches shutil.copytree(Path(source), cache_dir) # copy package to store in caches
Sources.init(cache_dir) # we need to run init command in directory where we do have permissions Sources.init(cache_dir) # we need to run init command in directory where we do have permissions
self.repository.database.build_queue_insert(package)
self.database.build_queue_insert(package)
self._process_dependencies(cache_dir, known_packages, without_dependencies) self._process_dependencies(cache_dir, known_packages, without_dependencies)
def _add_remote(self, source: str, *_: Any) -> None: def _add_remote(self, source: str, *_: Any) -> None:
""" """
add package from remote sources (e.g. HTTP) add package from remote sources (e.g. HTTP)
:param remote_url: remote URL to the package archive
Args:
source(str): remote URL of the package archive
""" """
dst = self.repository.paths.packages / Path(source).name # URL is path, is not it? dst = self.repository.paths.packages / Path(source).name # URL is path, is not it?
response = requests.get(source, stream=True) response = requests.get(source, stream=True)
@ -113,9 +135,11 @@ class Packages(Properties):
def _process_dependencies(self, local_path: Path, known_packages: Set[str], without_dependencies: bool) -> None: def _process_dependencies(self, local_path: Path, known_packages: Set[str], without_dependencies: bool) -> None:
""" """
process package dependencies process package dependencies
:param local_path: path to local package sources (i.e. cloned AUR repository)
:param known_packages: list of packages which are known by the service Args:
:param without_dependencies: if set, dependency check will be disabled local_path(Path): path to local package sources (i.e. cloned AUR repository)
known_packages(Set[str]): list of packages which are known by the service
without_dependencies(bool): if set, dependency check will be disabled
""" """
if without_dependencies: if without_dependencies:
return return
@ -126,9 +150,11 @@ class Packages(Properties):
def add(self, names: Iterable[str], source: PackageSource, without_dependencies: bool) -> None: def add(self, names: Iterable[str], source: PackageSource, without_dependencies: bool) -> None:
""" """
add packages for the next build add packages for the next build
:param names: list of package bases to add
:param source: package source to add Args:
:param without_dependencies: if set, dependency check will be disabled names(Iterable[str]): list of package bases to add
source(PackageSource): package source to add
without_dependencies(bool): if set, dependency check will be disabled
""" """
known_packages = self._known_packages() # speedup dependencies processing known_packages = self._known_packages() # speedup dependencies processing
@ -140,7 +166,9 @@ class Packages(Properties):
def remove(self, names: Iterable[str]) -> None: def remove(self, names: Iterable[str]) -> None:
""" """
remove packages from repository remove packages from repository
:param names: list of packages (either base or name) to remove
Args:
names(Iterable[str]): list of packages (either base or name) to remove
""" """
self.repository.process_remove(names) self.repository.process_remove(names)
self._finalize(Result()) self._finalize(Result())

View File

@ -27,20 +27,24 @@ from ahriman.core.repository import Repository
class Properties: class Properties:
""" """
application base properties class application base properties class
:ivar architecture: repository architecture
:ivar configuration: configuration instance Attributes:
:ivar database: database instance architecture(str): repository architecture
:ivar logger: application logger configuration(Configuration): configuration instance
:ivar repository: repository instance database(SQLite): database instance
logger(logging.Logger): application logger
repository(Repository): repository instance
""" """
def __init__(self, architecture: str, configuration: Configuration, no_report: bool, unsafe: bool) -> None: def __init__(self, architecture: str, configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture
:param configuration: configuration instance Args:
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
self.logger = logging.getLogger("root") self.logger = logging.getLogger("root")
self.configuration = configuration self.configuration = configuration

View File

@ -38,17 +38,24 @@ class Repository(Properties):
def _finalize(self, result: Result) -> None: def _finalize(self, result: Result) -> None:
""" """
generate report and sync to remote server generate report and sync to remote server
:param result: build result
Args:
result(Result): build result
Raises:
NotImplementedError: not implemented method
""" """
raise NotImplementedError raise NotImplementedError
def clean(self, cache: bool, chroot: bool, manual: bool, packages: bool) -> None: def clean(self, cache: bool, chroot: bool, manual: bool, packages: bool) -> None:
""" """
run all clean methods. Warning: some functions might not be available under non-root run all clean methods. Warning: some functions might not be available under non-root
:param cache: clear directory with package caches
:param chroot: clear build chroot Args:
:param manual: clear directory with manually added packages cache(bool): clear directory with package caches
:param packages: clear directory with built packages chroot(bool): clear build chroot
manual(bool): clear directory with manually added packages
packages(bool): clear directory with built packages
""" """
if cache: if cache:
self.repository.clear_cache() self.repository.clear_cache()
@ -62,8 +69,10 @@ class Repository(Properties):
def report(self, target: Iterable[str], result: Result) -> None: def report(self, target: Iterable[str], result: Result) -> None:
""" """
generate report generate report
:param target: list of targets to run (e.g. html)
:param result: build result Args:
target(Iterable[str]): list of targets to run (e.g. html)
result(Result): build result
""" """
targets = target or None targets = target or None
self.repository.process_report(targets, result) self.repository.process_report(targets, result)
@ -71,7 +80,9 @@ class Repository(Properties):
def sign(self, packages: Iterable[str]) -> None: def sign(self, packages: Iterable[str]) -> None:
""" """
sign packages and repository sign packages and repository
:param packages: only sign specified packages
Args:
packages(Iterable[str]): only sign specified packages
""" """
# copy to prebuilt directory # copy to prebuilt directory
for package in self.repository.packages(): for package in self.repository.packages():
@ -94,8 +105,10 @@ class Repository(Properties):
def sync(self, target: Iterable[str], built_packages: Iterable[Package]) -> None: def sync(self, target: Iterable[str], built_packages: Iterable[Package]) -> None:
""" """
sync to remote server sync to remote server
:param target: list of targets to run (e.g. s3)
:param built_packages: list of packages which has just been built Args:
target(Iterable[str]): list of targets to run (e.g. s3)
built_packages(Iterable[Package]): list of packages which has just been built
""" """
targets = target or None targets = target or None
self.repository.process_sync(targets, built_packages) self.repository.process_sync(targets, built_packages)
@ -103,7 +116,9 @@ class Repository(Properties):
def unknown(self) -> List[str]: def unknown(self) -> List[str]:
""" """
get packages which were not found in AUR get packages which were not found in AUR
:return: unknown package archive list
Returns:
List[str]: unknown package archive list
""" """
def has_local(probe: Package) -> bool: def has_local(probe: Package) -> bool:
cache_dir = self.repository.paths.cache_for(probe.base) cache_dir = self.repository.paths.cache_for(probe.base)
@ -135,7 +150,12 @@ class Repository(Properties):
def update(self, updates: Iterable[Package]) -> Result: def update(self, updates: Iterable[Package]) -> Result:
""" """
run package updates run package updates
:param updates: list of packages to update
Args:
updates(Iterable[Package]): list of packages to update
Returns:
Result: update result
""" """
def process_update(paths: Iterable[Path], result: Result) -> None: def process_update(paths: Iterable[Path], result: Result) -> None:
if not paths: if not paths:
@ -162,13 +182,17 @@ class Repository(Properties):
log_fn: Callable[[str], None]) -> List[Package]: log_fn: Callable[[str], None]) -> List[Package]:
""" """
get list of packages to run update process get list of packages to run update process
:param filter_packages: do not check every package just specified in the list
:param no_aur: do not check for aur updates Args:
:param no_local: do not check local packages for updates filter_packages(Iterable[str]): do not check every package just specified in the list
:param no_manual: do not check for manual updates no_aur(bool): do not check for aur updates
:param no_vcs: do not check VCS packages no_local(bool): do not check local packages for updates
:param log_fn: logger function to log updates no_manual(bool): do not check for manual updates
:return: list of out-of-dated packages no_vcs(bool): do not check VCS packages
log_fn(Callable[[str], None]): logger function to log updates
Returns:
List[Package]: list of out-of-dated packages
""" """
updates = {} updates = {}

View File

@ -20,6 +20,7 @@
from ahriman.application.handlers.handler import Handler from ahriman.application.handlers.handler import Handler
from ahriman.application.handlers.add import Add from ahriman.application.handlers.add import Add
from ahriman.application.handlers.backup import Backup
from ahriman.application.handlers.clean import Clean from ahriman.application.handlers.clean import Clean
from ahriman.application.handlers.dump import Dump from ahriman.application.handlers.dump import Dump
from ahriman.application.handlers.help import Help from ahriman.application.handlers.help import Help
@ -29,6 +30,7 @@ from ahriman.application.handlers.rebuild import Rebuild
from ahriman.application.handlers.remove import Remove from ahriman.application.handlers.remove import Remove
from ahriman.application.handlers.remove_unknown import RemoveUnknown from ahriman.application.handlers.remove_unknown import RemoveUnknown
from ahriman.application.handlers.report import Report from ahriman.application.handlers.report import Report
from ahriman.application.handlers.restore import Restore
from ahriman.application.handlers.search import Search from ahriman.application.handlers.search import Search
from ahriman.application.handlers.setup import Setup from ahriman.application.handlers.setup import Setup
from ahriman.application.handlers.sign import Sign from ahriman.application.handlers.sign import Sign

View File

@ -36,11 +36,13 @@ class Add(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
application = Application(architecture, configuration, no_report, unsafe) application = Application(architecture, configuration, no_report, unsafe)
application.add(args.package, args.source, args.without_dependencies) application.add(args.package, args.source, args.without_dependencies)

View File

@ -0,0 +1,86 @@
#
# Copyright (c) 2021-2022 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
import pwd
from pathlib import Path
from tarfile import TarFile
from typing import Set, Type
from ahriman.application.handlers.handler import Handler
from ahriman.core.configuration import Configuration
from ahriman.core.database.sqlite import SQLite
class Backup(Handler):
"""
backup packages handler
"""
ALLOW_AUTO_ARCHITECTURE_RUN = False # it should be called only as "no-architecture"
@classmethod
def run(cls: Type[Handler], args: argparse.Namespace, architecture: str,
configuration: Configuration, no_report: bool, unsafe: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
architecture(str): repository architecture
configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
backup_paths = Backup.get_paths(configuration)
with TarFile(args.path, mode="w") as archive: # well we don't actually use compression
for backup_path in backup_paths:
archive.add(backup_path)
@staticmethod
def get_paths(configuration: Configuration) -> Set[Path]:
"""
extract paths to backup
Args:
configuration(Configuration): configuration instance
Returns:
Set[Path]: map of the filesystem paths
"""
paths = set(configuration.include.glob("*.ini"))
root, _ = configuration.check_loaded()
paths.add(root) # the configuration itself
paths.add(SQLite.database_path(configuration)) # database
# local caches
repository_paths = configuration.repository_paths
if repository_paths.cache.is_dir():
paths.add(repository_paths.cache)
# gnupg home with imported keys
uid, _ = repository_paths.root_owner
system_user = pwd.getpwuid(uid)
gnupg_home = Path(system_user.pw_dir) / ".gnupg"
if gnupg_home.is_dir():
paths.add(gnupg_home)
return paths

View File

@ -36,11 +36,13 @@ class Clean(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
Application(architecture, configuration, no_report, unsafe).clean( Application(architecture, configuration, no_report, unsafe).clean(
args.cache, args.chroot, args.manual, args.packages) args.cache, args.chroot, args.manual, args.packages)

View File

@ -38,11 +38,13 @@ class Dump(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
dump = configuration.dump() dump = configuration.dump()
for section, values in sorted(dump.items()): for section, values in sorted(dump.items()):

View File

@ -34,8 +34,10 @@ from ahriman.models.repository_paths import RepositoryPaths
class Handler: class Handler:
""" """
base handler class for command callbacks base handler class for command callbacks
:cvar ALLOW_AUTO_ARCHITECTURE_RUN: allow to define architecture from existing repositories
:cvar ALLOW_MULTI_ARCHITECTURE_RUN: allow to run with multiple architectures Attributes:
ALLOW_AUTO_ARCHITECTURE_RUN(bool): (class attribute) allow defining architecture from existing repositories
ALLOW_MULTI_ARCHITECTURE_RUN(bool): (class attribute) allow running with multiple architectures
""" """
ALLOW_AUTO_ARCHITECTURE_RUN = True ALLOW_AUTO_ARCHITECTURE_RUN = True
@ -45,8 +47,15 @@ class Handler:
def architectures_extract(cls: Type[Handler], args: argparse.Namespace) -> List[str]: def architectures_extract(cls: Type[Handler], args: argparse.Namespace) -> List[str]:
""" """
get known architectures get known architectures
:param args: command line args
:return: list of architectures for which tree is created Args:
args(argparse.Namespace): command line args
Returns:
List[str]: list of architectures for which tree is created
Raises:
MissingArchitecture: if no architecture set and automatic detection is not allowed or failed
""" """
if not cls.ALLOW_AUTO_ARCHITECTURE_RUN and args.architecture is None: if not cls.ALLOW_AUTO_ARCHITECTURE_RUN and args.architecture is None:
# for some parsers (e.g. config) we need to run with specific architecture # for some parsers (e.g. config) we need to run with specific architecture
@ -69,9 +78,13 @@ class Handler:
def call(cls: Type[Handler], args: argparse.Namespace, architecture: str) -> bool: def call(cls: Type[Handler], args: argparse.Namespace, architecture: str) -> bool:
""" """
additional function to wrap all calls for multiprocessing library additional function to wrap all calls for multiprocessing library
:param args: command line args
:param architecture: repository architecture Args:
:return: True on success, False otherwise args(argparse.Namespace): command line args
architecture(str): repository architecture
Returns:
bool: True on success, False otherwise
""" """
try: try:
configuration = Configuration.from_path(args.configuration, architecture, args.quiet) configuration = Configuration.from_path(args.configuration, architecture, args.quiet)
@ -89,8 +102,15 @@ class Handler:
def execute(cls: Type[Handler], args: argparse.Namespace) -> int: def execute(cls: Type[Handler], args: argparse.Namespace) -> int:
""" """
execute function for all aru execute function for all aru
:param args: command line args
:return: 0 on success, 1 otherwise Args:
args(argparse.Namespace): command line args
Returns:
int: 0 on success, 1 otherwise
Raises:
MultipleArchitectures: if more than one architecture supplied and no multi architecture supported
""" """
architectures = cls.architectures_extract(args) architectures = cls.architectures_extract(args)
@ -112,11 +132,16 @@ class Handler:
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
Raises:
NotImplementedError: not implemented method
""" """
raise NotImplementedError raise NotImplementedError
@ -124,8 +149,13 @@ class Handler:
def check_if_empty(enabled: bool, predicate: bool) -> None: def check_if_empty(enabled: bool, predicate: bool) -> None:
""" """
check condition and flag and raise ExitCode exception in case if it is enabled and condition match check condition and flag and raise ExitCode exception in case if it is enabled and condition match
:param enabled: if False no check will be performed
:param predicate: indicates condition on which exception should be thrown Args:
enabled(bool): if False no check will be performed
predicate(bool): indicates condition on which exception should be thrown
Raises:
ExitCode: if result is empty and check is enabled
""" """
if enabled and predicate: if enabled and predicate:
raise ExitCode() raise ExitCode()

View File

@ -37,11 +37,13 @@ class Help(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
parser: argparse.ArgumentParser = args.parser() parser: argparse.ArgumentParser = args.parser()
if args.command is None: if args.command is None:

View File

@ -38,11 +38,13 @@ class KeyImport(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
Application(architecture, configuration, no_report, unsafe).repository.sign.key_import( Application(architecture, configuration, no_report, unsafe).repository.sign.key_import(
args.key_server, args.key) args.key_server, args.key)

View File

@ -42,11 +42,13 @@ class Patch(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
application = Application(architecture, configuration, no_report, unsafe) application = Application(architecture, configuration, no_report, unsafe)
@ -61,9 +63,11 @@ class Patch(Handler):
def patch_set_create(application: Application, sources_dir: str, track: List[str]) -> None: def patch_set_create(application: Application, sources_dir: str, track: List[str]) -> None:
""" """
create patch set for the package base create patch set for the package base
:param application: application instance
:param sources_dir: path to directory with the package sources Args:
:param track: track files which match the glob before creating the patch application(Application): application instance
sources_dir(str): path to directory with the package sources
track(List[str]): track files which match the glob before creating the patch
""" """
package = Package.load(sources_dir, PackageSource.Local, application.repository.pacman, package = Package.load(sources_dir, PackageSource.Local, application.repository.pacman,
application.repository.aur_url) application.repository.aur_url)
@ -74,9 +78,11 @@ class Patch(Handler):
def patch_set_list(application: Application, package_base: Optional[str], exit_code: bool) -> None: def patch_set_list(application: Application, package_base: Optional[str], exit_code: bool) -> None:
""" """
list patches available for the package base list patches available for the package base
:param application: application instance
:param package_base: package base Args:
:param exit_code: raise ExitCode on empty search result application(Application): application instance
package_base(Optional[str]): package base
exit_code(bool): exit with error on empty search result
""" """
patches = application.database.patches_list(package_base) patches = application.database.patches_list(package_base)
Patch.check_if_empty(exit_code, not patches) Patch.check_if_empty(exit_code, not patches)
@ -89,7 +95,9 @@ class Patch(Handler):
def patch_set_remove(application: Application, package_base: str) -> None: def patch_set_remove(application: Application, package_base: str) -> None:
""" """
remove patch set for the package base remove patch set for the package base
:param application: application instance
:param package_base: package base Args:
application(Application): application instance
package_base(str): package base
""" """
application.database.patches_remove(package_base) application.database.patches_remove(package_base)

View File

@ -19,12 +19,13 @@
# #
import argparse import argparse
from typing import Type from typing import List, Type
from ahriman.application.application import Application from ahriman.application.application import Application
from ahriman.application.handlers.handler import Handler from ahriman.application.handlers.handler import Handler
from ahriman.core.configuration import Configuration from ahriman.core.configuration import Configuration
from ahriman.core.formatters.update_printer import UpdatePrinter from ahriman.core.formatters.update_printer import UpdatePrinter
from ahriman.models.package import Package
class Rebuild(Handler): class Rebuild(Handler):
@ -37,16 +38,21 @@ class Rebuild(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
depends_on = set(args.depends_on) if args.depends_on else None depends_on = set(args.depends_on) if args.depends_on else None
application = Application(architecture, configuration, no_report, unsafe) application = Application(architecture, configuration, no_report, unsafe)
updates = application.repository.packages_depends_on(depends_on) if args.from_database:
updates = Rebuild.extract_packages(application)
else:
updates = application.repository.packages_depends_on(depends_on)
Rebuild.check_if_empty(args.exit_code, not updates) Rebuild.check_if_empty(args.exit_code, not updates)
if args.dry_run: if args.dry_run:
@ -56,3 +62,16 @@ class Rebuild(Handler):
result = application.update(updates) result = application.update(updates)
Rebuild.check_if_empty(args.exit_code, result.is_empty) Rebuild.check_if_empty(args.exit_code, result.is_empty)
@staticmethod
def extract_packages(application: Application) -> List[Package]:
"""
extract packages from database file
Args:
application(Application): application instance
Returns:
List[Package]: list of packages which were stored in database
"""
return [package for (package, _) in application.database.packages_get()]

View File

@ -36,10 +36,12 @@ class Remove(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
Application(architecture, configuration, no_report, unsafe).remove(args.package) Application(architecture, configuration, no_report, unsafe).remove(args.package)

View File

@ -37,11 +37,13 @@ class RemoveUnknown(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
application = Application(architecture, configuration, no_report, unsafe) application = Application(architecture, configuration, no_report, unsafe)
unknown_packages = application.unknown() unknown_packages = application.unknown()

View File

@ -37,10 +37,12 @@ class Report(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
Application(architecture, configuration, no_report, unsafe).report(args.target, Result()) Application(architecture, configuration, no_report, unsafe).report(args.target, Result())

View File

@ -0,0 +1,50 @@
#
# Copyright (c) 2021-2022 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from typing import Type
from tarfile import TarFile
from ahriman.application.handlers.handler import Handler
from ahriman.core.configuration import Configuration
class Restore(Handler):
"""
restore packages handler
"""
ALLOW_AUTO_ARCHITECTURE_RUN = False # it should be called only as "no-architecture"
@classmethod
def run(cls: Type[Handler], args: argparse.Namespace, architecture: str,
configuration: Configuration, no_report: bool, unsafe: bool) -> None:
"""
callback for command line
Args:
args(argparse.Namespace): command line args
architecture(str): repository architecture
configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
with TarFile(args.path) as archive:
archive.extractall(path=args.output)

View File

@ -23,7 +23,8 @@ from dataclasses import fields
from typing import Callable, Iterable, List, Tuple, Type from typing import Callable, Iterable, List, Tuple, Type
from ahriman.application.handlers.handler import Handler from ahriman.application.handlers.handler import Handler
from ahriman.core.alpm.aur import AUR from ahriman.core.alpm.remote.aur import AUR
from ahriman.core.alpm.remote.official import Official
from ahriman.core.configuration import Configuration from ahriman.core.configuration import Configuration
from ahriman.core.exceptions import InvalidOption from ahriman.core.exceptions import InvalidOption
from ahriman.core.formatters.aur_printer import AurPrinter from ahriman.core.formatters.aur_printer import AurPrinter
@ -33,7 +34,9 @@ from ahriman.models.aur_package import AURPackage
class Search(Handler): class Search(Handler):
""" """
packages search handler packages search handler
:cvar SORT_FIELDS: allowed fields to sort the package list
Attributes:
SORT_FIELDS(Set[str]): (class attribute) allowed fields to sort the package list
""" """
ALLOW_AUTO_ARCHITECTURE_RUN = False # it should be called only as "no-architecture" ALLOW_AUTO_ARCHITECTURE_RUN = False # it should be called only as "no-architecture"
@ -44,24 +47,37 @@ class Search(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
packages_list = AUR.multisearch(*args.search) official_packages_list = Official.multisearch(*args.search)
Search.check_if_empty(args.exit_code, not packages_list) aur_packages_list = AUR.multisearch(*args.search)
for package in Search.sort(packages_list, args.sort_by): Search.check_if_empty(args.exit_code, not official_packages_list and not aur_packages_list)
AurPrinter(package).print(args.info)
for packages_list in (official_packages_list, aur_packages_list):
# keep sorting by packages source
for package in Search.sort(packages_list, args.sort_by):
AurPrinter(package).print(args.info)
@staticmethod @staticmethod
def sort(packages: Iterable[AURPackage], sort_by: str) -> List[AURPackage]: def sort(packages: Iterable[AURPackage], sort_by: str) -> List[AURPackage]:
""" """
sort package list by specified field sort package list by specified field
:param packages: packages list to sort
:param sort_by: AUR package field name to sort by Args:
:return: sorted list for packages packages(Iterable[AURPackage]): packages list to sort
sort_by(str): AUR package field name to sort by
Returns:
List[AURPackage]: sorted list for packages
Raises:
InvalidOption: if search fields is not in list of allowed ones
""" """
if sort_by not in Search.SORT_FIELDS: if sort_by not in Search.SORT_FIELDS:
raise InvalidOption(sort_by) raise InvalidOption(sort_by)

View File

@ -31,10 +31,12 @@ from ahriman.models.repository_paths import RepositoryPaths
class Setup(Handler): class Setup(Handler):
""" """
setup handler setup handler
:cvar ARCHBUILD_COMMAND_PATH: default devtools command
:cvar BIN_DIR_PATH: directory for custom binaries Attributes:
:cvar MIRRORLIST_PATH: path to pacman default mirrorlist (used by multilib repository) ARCHBUILD_COMMAND_PATH(Path): (class attribute) default devtools command
:cvar SUDOERS_PATH: path to sudoers.d include configuration BIN_DIR_PATH(Path): (class attribute) directory for custom binaries
MIRRORLIST_PATH(Path): (class attribute) path to pacman default mirrorlist (used by multilib repository)
SUDOERS_PATH(Path): (class attribute) path to sudoers.d include configuration
""" """
ALLOW_AUTO_ARCHITECTURE_RUN = False ALLOW_AUTO_ARCHITECTURE_RUN = False
@ -49,11 +51,13 @@ class Setup(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
Setup.configuration_create_ahriman(args, architecture, args.repository, configuration.include) Setup.configuration_create_ahriman(args, architecture, args.repository, configuration.include)
configuration.reload() configuration.reload()
@ -72,9 +76,13 @@ class Setup(Handler):
def build_command(prefix: str, architecture: str) -> Path: def build_command(prefix: str, architecture: str) -> Path:
""" """
generate build command name generate build command name
:param prefix: command prefix in {prefix}-{architecture}-build
:param architecture: repository architecture Args:
:return: valid devtools command name prefix(str): command prefix in {prefix}-{architecture}-build
architecture(str): repository architecture
Returns:
Path: valid devtools command name
""" """
return Setup.BIN_DIR_PATH / f"{prefix}-{architecture}-build" return Setup.BIN_DIR_PATH / f"{prefix}-{architecture}-build"
@ -83,10 +91,12 @@ class Setup(Handler):
include_path: Path) -> None: include_path: Path) -> None:
""" """
create service specific configuration create service specific configuration
:param args: command line args
:param architecture: repository architecture Args:
:param repository: repository name args(argparse.Namespace): command line args
:param include_path: path to directory with configuration includes architecture(str): repository architecture
repository(str): repository name
include_path(Path): path to directory with configuration includes
""" """
configuration = Configuration() configuration = Configuration()
@ -114,12 +124,14 @@ class Setup(Handler):
no_multilib: bool, repository: str, paths: RepositoryPaths) -> None: no_multilib: bool, repository: str, paths: RepositoryPaths) -> None:
""" """
create configuration for devtools based on `source` configuration create configuration for devtools based on `source` configuration
:param prefix: command prefix in {prefix}-{architecture}-build
:param architecture: repository architecture Args:
:param source: path to source configuration file prefix(str): command prefix in {prefix}-{architecture}-build
:param no_multilib: do not add multilib repository architecture(str): repository architecture
:param repository: repository name source(Path): path to source configuration file
:param paths: repository paths instance no_multilib(bool): do not add multilib repository
repository(str): repository name
paths(RepositoryPaths): repository paths instance
""" """
configuration = Configuration() configuration = Configuration()
# preserve case # preserve case
@ -149,8 +161,10 @@ class Setup(Handler):
def configuration_create_makepkg(packager: str, paths: RepositoryPaths) -> None: def configuration_create_makepkg(packager: str, paths: RepositoryPaths) -> None:
""" """
create configuration for makepkg create configuration for makepkg
:param packager: packager identifier (e.g. name, email)
:param paths: repository paths instance Args:
packager(str): packager identifier (e.g. name, email)
paths(RepositoryPaths): repository paths instance
""" """
(paths.root / ".makepkg.conf").write_text(f"PACKAGER='{packager}'\n", encoding="utf8") (paths.root / ".makepkg.conf").write_text(f"PACKAGER='{packager}'\n", encoding="utf8")
@ -158,8 +172,10 @@ class Setup(Handler):
def configuration_create_sudo(prefix: str, architecture: str) -> None: def configuration_create_sudo(prefix: str, architecture: str) -> None:
""" """
create configuration to run build command with sudo without password create configuration to run build command with sudo without password
:param prefix: command prefix in {prefix}-{architecture}-build
:param architecture: repository architecture Args:
prefix(str): command prefix in {prefix}-{architecture}-build
architecture(str): repository architecture
""" """
command = Setup.build_command(prefix, architecture) command = Setup.build_command(prefix, architecture)
Setup.SUDOERS_PATH.write_text(f"ahriman ALL=(ALL) NOPASSWD: {command} *\n", encoding="utf8") Setup.SUDOERS_PATH.write_text(f"ahriman ALL=(ALL) NOPASSWD: {command} *\n", encoding="utf8")
@ -169,8 +185,10 @@ class Setup(Handler):
def executable_create(prefix: str, architecture: str) -> None: def executable_create(prefix: str, architecture: str) -> None:
""" """
create executable for the service create executable for the service
:param prefix: command prefix in {prefix}-{architecture}-build
:param architecture: repository architecture Args:
prefix(str): command prefix in {prefix}-{architecture}-build
architecture(str): repository architecture
""" """
command = Setup.build_command(prefix, architecture) command = Setup.build_command(prefix, architecture)
command.unlink(missing_ok=True) command.unlink(missing_ok=True)

View File

@ -36,10 +36,12 @@ class Sign(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
Application(architecture, configuration, no_report, unsafe).sign(args.package) Application(architecture, configuration, no_report, unsafe).sign(args.package)

View File

@ -42,11 +42,13 @@ class Status(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
# we are using reporter here # we are using reporter here
client = Application(architecture, configuration, no_report=False, unsafe=unsafe).repository.reporter client = Application(architecture, configuration, no_report=False, unsafe=unsafe).repository.reporter

View File

@ -39,11 +39,13 @@ class StatusUpdate(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
# we are using reporter here # we are using reporter here
client = Application(architecture, configuration, no_report=False, unsafe=unsafe).repository.reporter client = Application(architecture, configuration, no_report=False, unsafe=unsafe).repository.reporter

View File

@ -28,7 +28,7 @@ from ahriman.core.configuration import Configuration
class Sync(Handler): class Sync(Handler):
""" """
remove sync handler remote sync handler
""" """
@classmethod @classmethod
@ -36,10 +36,12 @@ class Sync(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
Application(architecture, configuration, no_report, unsafe).sync(args.target, []) Application(architecture, configuration, no_report, unsafe).sync(args.target, [])

View File

@ -24,7 +24,6 @@ from typing import List, Type
from ahriman.application.handlers.handler import Handler from ahriman.application.handlers.handler import Handler
from ahriman.core.configuration import Configuration from ahriman.core.configuration import Configuration
from ahriman.core.exceptions import ExitCode
from ahriman.core.formatters.string_printer import StringPrinter from ahriman.core.formatters.string_printer import StringPrinter
@ -40,11 +39,13 @@ class UnsafeCommands(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
parser = args.parser() parser = args.parser()
unsafe_commands = UnsafeCommands.get_unsafe_commands(parser) unsafe_commands = UnsafeCommands.get_unsafe_commands(parser)
@ -58,20 +59,25 @@ class UnsafeCommands(Handler):
def check_unsafe(command: str, unsafe_commands: List[str], parser: argparse.ArgumentParser) -> None: def check_unsafe(command: str, unsafe_commands: List[str], parser: argparse.ArgumentParser) -> None:
""" """
check if command is unsafe check if command is unsafe
:param command: command to check
:param unsafe_commands: list of unsafe commands Args:
:param parser: generated argument parser command(str): command to check
unsafe_commands(List[str]): list of unsafe commands
parser(argparse.ArgumentParser): generated argument parser
""" """
args = parser.parse_args(shlex.split(command)) args = parser.parse_args(shlex.split(command))
if args.command in unsafe_commands: UnsafeCommands.check_if_empty(True, args.command in unsafe_commands)
raise ExitCode()
@staticmethod @staticmethod
def get_unsafe_commands(parser: argparse.ArgumentParser) -> List[str]: def get_unsafe_commands(parser: argparse.ArgumentParser) -> List[str]:
""" """
extract unsafe commands from argument parser extract unsafe commands from argument parser
:param parser: generated argument parser
:return: list of commands with default unsafe flag Args:
parser(argparse.ArgumentParser): generated argument parser
Returns:
List[str]: list of commands with default unsafe flag
""" """
# pylint: disable=protected-access # pylint: disable=protected-access
subparser = next(action for action in parser._actions if isinstance(action, argparse._SubParsersAction)) subparser = next(action for action in parser._actions if isinstance(action, argparse._SubParsersAction))

View File

@ -36,11 +36,13 @@ class Update(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
application = Application(architecture, configuration, no_report, unsafe) application = Application(architecture, configuration, no_report, unsafe)
packages = application.updates(args.package, args.no_aur, args.no_local, args.no_manual, args.no_vcs, packages = application.updates(args.package, args.no_aur, args.no_local, args.no_manual, args.no_vcs,
@ -56,9 +58,13 @@ class Update(Handler):
def log_fn(application: Application, dry_run: bool) -> Callable[[str], None]: def log_fn(application: Application, dry_run: bool) -> Callable[[str], None]:
""" """
package updates log function package updates log function
:param application: application instance
:param dry_run: do not perform update itself Args:
:return: in case if dry_run is set it will return print, logger otherwise application(Application): application instance
dry_run(bool): do not perform update itself
Returns:
Callable[[str], None]: in case if dry_run is set it will return print, logger otherwise
""" """
def inner(line: str) -> None: def inner(line: str) -> None:
return print(line) if dry_run else application.logger.info(line) return print(line) if dry_run else application.logger.info(line)

View File

@ -43,11 +43,13 @@ class User(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
database = SQLite.load(configuration) database = SQLite.load(configuration)
@ -72,11 +74,13 @@ class User(Handler):
as_service_user: bool, secure: bool) -> None: as_service_user: bool, secure: bool) -> None:
""" """
enable configuration if it has been disabled enable configuration if it has been disabled
:param configuration: configuration instance
:param user: user descriptor Args:
:param salt: password hash salt configuration(Configuration): configuration instance
:param as_service_user: add user as service user, also set password and user to configuration user(MUser): user descriptor
:param secure: if true then set file permissions to 0o600 salt(str): password hash salt
as_service_user(bool): add user as service user, also set password and user to configuration
secure(bool): if true then set file permissions to 0o600
""" """
configuration.set_option("auth", "salt", salt) configuration.set_option("auth", "salt", salt)
if as_service_user: if as_service_user:
@ -88,8 +92,12 @@ class User(Handler):
def configuration_get(include_path: Path) -> Configuration: def configuration_get(include_path: Path) -> Configuration:
""" """
create configuration instance create configuration instance
:param include_path: path to directory with configuration includes
:return: configuration instance. In case if there are local settings they will be loaded Args:
include_path(Path): path to directory with configuration includes
Returns:
Configuration: configuration instance. In case if there are local settings they will be loaded
""" """
target = include_path / "auth.ini" target = include_path / "auth.ini"
configuration = Configuration() configuration = Configuration()
@ -103,8 +111,10 @@ class User(Handler):
def configuration_write(configuration: Configuration, secure: bool) -> None: def configuration_write(configuration: Configuration, secure: bool) -> None:
""" """
write configuration file write configuration file
:param configuration: configuration instance
:param secure: if true then set file permissions to 0o600 Args:
configuration(Configuration): configuration instance
secure(bool): if true then set file permissions to 0o600
""" """
path, _ = configuration.check_loaded() path, _ = configuration.check_loaded()
with path.open("w") as ahriman_configuration: with path.open("w") as ahriman_configuration:
@ -116,9 +126,13 @@ class User(Handler):
def get_salt(configuration: Configuration, salt_length: int = 20) -> str: def get_salt(configuration: Configuration, salt_length: int = 20) -> str:
""" """
get salt from configuration or create new string get salt from configuration or create new string
:param configuration: configuration instance
:param salt_length: salt length Args:
:return: current salt configuration(Configuration): configuration instance
salt_length(int, optional): salt length (Default value = 20)
Returns:
str: current salt
""" """
if salt := configuration.get("auth", "salt", fallback=None): if salt := configuration.get("auth", "salt", fallback=None):
return salt return salt
@ -128,8 +142,12 @@ class User(Handler):
def user_create(args: argparse.Namespace) -> MUser: def user_create(args: argparse.Namespace) -> MUser:
""" """
create user descriptor from arguments create user descriptor from arguments
:param args: command line args
:return: built user descriptor Args:
args(argparse.Namespace): command line args
Returns:
MUser: built user descriptor
""" """
user = MUser(args.username, args.password, args.role) user = MUser(args.username, args.password, args.role)
if user.password is None: if user.password is None:

View File

@ -39,11 +39,13 @@ class Web(Handler):
configuration: Configuration, no_report: bool, unsafe: bool) -> None: configuration: Configuration, no_report: bool, unsafe: bool) -> None:
""" """
callback for command line callback for command line
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
:param no_report: force disable reporting architecture(str): repository architecture
:param unsafe: if set no user check will be performed before path creation configuration(Configuration): configuration instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
# we are using local import for optional dependencies # we are using local import for optional dependencies
from ahriman.web.web import run_server, setup_service from ahriman.web.web import run_server, setup_service

View File

@ -37,19 +37,23 @@ from ahriman.models.build_status import BuildStatusEnum
class Lock: class Lock:
""" """
wrapper for application lock file wrapper for application lock file
:ivar force: remove lock file on start if any
:ivar path: path to lock file if any Attributes:
:ivar reporter: build status reporter instance force(bool): remove lock file on start if any
:ivar paths: repository paths instance path(Path): path to lock file if any
:ivar unsafe: skip user check reporter(Client): build status reporter instance
paths(RepositoryPaths): repository paths instance
unsafe(bool): skip user check
""" """
def __init__(self, args: argparse.Namespace, architecture: str, configuration: Configuration) -> None: def __init__(self, args: argparse.Namespace, architecture: str, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param args: command line args
:param architecture: repository architecture Args:
:param configuration: configuration instance args(argparse.Namespace): command line args
architecture(str): repository architecture
configuration(Configuration): configuration instance
""" """
self.path = Path(f"{args.lock}_{architecture}") if args.lock is not None else None self.path = Path(f"{args.lock}_{architecture}") if args.lock is not None else None
self.force = args.force self.force = args.force
@ -62,11 +66,11 @@ class Lock:
""" """
default workflow is the following: default workflow is the following:
check user UID 1. Check user UID
check if there is lock file 2. Check if there is lock file
check web status watcher status 3. Check web status watcher status
create lock file 4. Create lock file
report to web if enabled 5. Report to status page if enabled
""" """
self.check_user() self.check_user()
self.check_version() self.check_version()
@ -78,10 +82,14 @@ class Lock:
exc_tb: TracebackType) -> Literal[False]: exc_tb: TracebackType) -> Literal[False]:
""" """
remove lock file when done remove lock file when done
:param exc_type: exception type name if any
:param exc_val: exception raised if any Args:
:param exc_tb: exception traceback if any exc_type(Optional[Type[Exception]]): exception type name if any
:return: always False (do not suppress any exception) exc_val(Optional[Exception]): exception raised if any
exc_tb(TracebackType): exception traceback if any
Returns:
Literal[False]: always False (do not suppress any exception)
""" """
self.clear() self.clear()
status = BuildStatusEnum.Success if exc_val is None else BuildStatusEnum.Failed status = BuildStatusEnum.Success if exc_val is None else BuildStatusEnum.Failed
@ -116,6 +124,9 @@ class Lock:
def create(self) -> None: def create(self) -> None:
""" """
create lock file create lock file
Raises:
DuplicateRun: if lock exists and no force flag supplied
""" """
if self.path is None: if self.path is None:
return return

View File

@ -26,13 +26,17 @@ from ahriman.core.configuration import Configuration
class Pacman: class Pacman:
""" """
alpm wrapper alpm wrapper
:ivar handle: pyalpm root `Handle`
Attributes:
handle(Handle): pyalpm root `Handle`
""" """
def __init__(self, configuration: Configuration) -> None: def __init__(self, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param configuration: configuration instance
Args:
configuration(Configuration): configuration instance
""" """
root = configuration.get("alpm", "root") root = configuration.get("alpm", "root")
pacman_root = configuration.getpath("alpm", "database") pacman_root = configuration.getpath("alpm", "database")
@ -43,7 +47,9 @@ class Pacman:
def all_packages(self) -> Set[str]: def all_packages(self) -> Set[str]:
""" """
get list of packages known for alpm get list of packages known for alpm
:return: list of package names
Returns:
Set[str]: list of package names
""" """
result: Set[str] = set() result: Set[str] = set()
for database in self.handle.get_syncdbs(): for database in self.handle.get_syncdbs():

View File

@ -0,0 +1,19 @@
#
# Copyright (c) 2021-2022 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#

View File

@ -17,26 +17,25 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
# #
from __future__ import annotations
import logging
import requests import requests
from typing import Any, Dict, List, Optional, Type from typing import Any, Dict, List, Optional
from ahriman.core.alpm.remote.remote import Remote
from ahriman.core.exceptions import InvalidPackageInfo from ahriman.core.exceptions import InvalidPackageInfo
from ahriman.core.util import exception_response_text from ahriman.core.util import exception_response_text
from ahriman.models.aur_package import AURPackage from ahriman.models.aur_package import AURPackage
class AUR: class AUR(Remote):
""" """
AUR RPC wrapper AUR RPC wrapper
:cvar DEFAULT_RPC_URL: default AUR RPC url
:cvar DEFAULT_RPC_VERSION: default AUR RPC version Attributes:
:ivar logger: class logger DEFAULT_RPC_URL(str): (class attribute) default AUR RPC url
:ivar rpc_url: AUR RPC url DEFAULT_RPC_VERSION(str): (class attribute) default AUR RPC version
:ivar rpc_version: AUR RPC version rpc_url(str): AUR RPC url
rpc_version(str): AUR RPC version
""" """
DEFAULT_RPC_URL = "https://aur.archlinux.org/rpc" DEFAULT_RPC_URL = "https://aur.archlinux.org/rpc"
@ -45,56 +44,28 @@ class AUR:
def __init__(self, rpc_url: Optional[str] = None, rpc_version: Optional[str] = None) -> None: def __init__(self, rpc_url: Optional[str] = None, rpc_version: Optional[str] = None) -> None:
""" """
default constructor default constructor
:param rpc_url: AUR RPC url
:param rpc_version: AUR RPC version Args:
rpc_url(Optional[str], optional): AUR RPC url (Default value = None)
rpc_version(Optional[str], optional): AUR RPC version (Default value = None)
""" """
Remote.__init__(self)
self.rpc_url = rpc_url or self.DEFAULT_RPC_URL self.rpc_url = rpc_url or self.DEFAULT_RPC_URL
self.rpc_version = rpc_version or self.DEFAULT_RPC_VERSION self.rpc_version = rpc_version or self.DEFAULT_RPC_VERSION
self.logger = logging.getLogger("build_details")
@classmethod
def info(cls: Type[AUR], package_name: str) -> AURPackage:
"""
get package info by its name
:param package_name: package name to search
:return: package which match the package name
"""
return cls().package_info(package_name)
@classmethod
def multisearch(cls: Type[AUR], *keywords: str) -> List[AURPackage]:
"""
search in AUR by using API with multiple words. This method is required in order to handle
https://bugs.archlinux.org/task/49133. In addition short words will be dropped
:param keywords: search terms, e.g. "ahriman", "is", "cool"
:return: list of packages each of them matches all search terms
"""
instance = cls()
packages: Dict[str, AURPackage] = {}
for term in filter(lambda word: len(word) > 3, keywords):
portion = instance.search(term)
packages = {
package.package_base: package
for package in portion
if package.package_base in packages or not packages
}
return list(packages.values())
@classmethod
def search(cls: Type[AUR], *keywords: str) -> List[AURPackage]:
"""
search package in AUR web
:param keywords: keywords to search
:return: list of packages which match the criteria
"""
return cls().package_search(*keywords)
@staticmethod @staticmethod
def parse_response(response: Dict[str, Any]) -> List[AURPackage]: def parse_response(response: Dict[str, Any]) -> List[AURPackage]:
""" """
parse RPC response to package list parse RPC response to package list
:param response: RPC response json
:return: list of parsed packages Args:
response(Dict[str, Any]): RPC response json
Returns:
List[AURPackage]: list of parsed packages
Raises:
InvalidPackageInfo: for error API response
""" """
response_type = response["type"] response_type = response["type"]
if response_type == "error": if response_type == "error":
@ -105,10 +76,14 @@ class AUR:
def make_request(self, request_type: str, *args: str, **kwargs: str) -> List[AURPackage]: def make_request(self, request_type: str, *args: str, **kwargs: str) -> List[AURPackage]:
""" """
perform request to AUR RPC perform request to AUR RPC
:param request_type: AUR request type, e.g. search, info
:param args: list of arguments to be passed as args query parameter Args:
:param kwargs: list of additional named parameters like by request_type(str): AUR request type, e.g. search, info
:return: response parsed to package list *args(str): list of arguments to be passed as args query parameter
**kwargs(str): list of additional named parameters like by
Returns:
List[AURPackage]: response parsed to package list
""" """
query: Dict[str, Any] = { query: Dict[str, Any] = {
"type": request_type, "type": request_type,
@ -138,17 +113,24 @@ class AUR:
def package_info(self, package_name: str) -> AURPackage: def package_info(self, package_name: str) -> AURPackage:
""" """
get package info by its name get package info by its name
:param package_name: package name to search
:return: package which match the package name Args:
package_name(str): package name to search
Returns:
AURPackage: package which match the package name
""" """
packages = self.make_request("info", package_name) packages = self.make_request("info", package_name)
return next(package for package in packages if package.name == package_name) return next(package for package in packages if package.name == package_name)
def package_search(self, *keywords: str, by: str = "name-desc") -> List[AURPackage]: def package_search(self, *keywords: str) -> List[AURPackage]:
""" """
search package in AUR web search package in AUR web
:param keywords: keywords to search
:param by: search by the field Args:
:return: list of packages which match the criteria *keywords(str): keywords to search
Returns:
List[AURPackage]: list of packages which match the criteria
""" """
return self.make_request("search", *keywords, by=by) return self.make_request("search", *keywords, by="name-desc")

View File

@ -0,0 +1,114 @@
#
# Copyright (c) 2021-2022 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import requests
from typing import Any, Dict, List, Optional
from ahriman.core.alpm.remote.remote import Remote
from ahriman.core.exceptions import InvalidPackageInfo
from ahriman.core.util import exception_response_text
from ahriman.models.aur_package import AURPackage
class Official(Remote):
"""
official repository RPC wrapper
Attributes:
DEFAULT_RPC_URL(str): (class attribute) default AUR RPC url
rpc_url(str): AUR RPC url
"""
DEFAULT_RPC_URL = "https://archlinux.org/packages/search/json"
def __init__(self, rpc_url: Optional[str] = None) -> None:
"""
default constructor
Args:
rpc_url(Optional[str], optional): AUR RPC url (Default value = None)
"""
Remote.__init__(self)
self.rpc_url = rpc_url or self.DEFAULT_RPC_URL
@staticmethod
def parse_response(response: Dict[str, Any]) -> List[AURPackage]:
"""
parse RPC response to package list
Args:
response(Dict[str, Any]): RPC response json
Returns:
List[AURPackage]: list of parsed packages
Raises:
InvalidPackageInfo: for error API response
"""
if not response["valid"]:
raise InvalidPackageInfo("API validation error")
return [AURPackage.from_repo(package) for package in response["results"]]
def make_request(self, *args: str, by: str) -> List[AURPackage]:
"""
perform request to official repositories RPC
Args:
*args(str): list of arguments to be passed as args query parameter
by(str): search by the field
Returns:
List[AURPackage]: response parsed to package list
"""
try:
response = requests.get(self.rpc_url, params={by: args})
response.raise_for_status()
return self.parse_response(response.json())
except requests.HTTPError as e:
self.logger.exception("could not perform request: %s", exception_response_text(e))
raise
except Exception:
self.logger.exception("could not perform request")
raise
def package_info(self, package_name: str) -> AURPackage:
"""
get package info by its name
Args:
package_name(str): package name to search
Returns:
AURPackage: package which match the package name
"""
packages = self.make_request(package_name, by="name")
return next(package for package in packages if package.name == package_name)
def package_search(self, *keywords: str) -> List[AURPackage]:
"""
search package in AUR web
Args:
*keywords(str): keywords to search
Returns:
List[AURPackage]: list of packages which match the criteria
"""
return self.make_request(*keywords, by="q")

View File

@ -0,0 +1,120 @@
#
# Copyright (c) 2021-2022 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from __future__ import annotations
import logging
from typing import Dict, List, Type
from ahriman.models.aur_package import AURPackage
class Remote:
"""
base class for remote package search
Attributes:
logger(logging.Logger): class logger
"""
def __init__(self) -> None:
"""
default constructor
"""
self.logger = logging.getLogger("build_details")
@classmethod
def info(cls: Type[Remote], package_name: str) -> AURPackage:
"""
get package info by its name
Args:
package_name(str): package name to search
Returns:
AURPackage: package which match the package name
"""
return cls().package_info(package_name)
@classmethod
def multisearch(cls: Type[Remote], *keywords: str) -> List[AURPackage]:
"""
search in remote repository by using API with multiple words. This method is required in order to handle
https://bugs.archlinux.org/task/49133. In addition, short words will be dropped
Args:
*keywords(str): search terms, e.g. "ahriman", "is", "cool"
Returns:
List[AURPackage]: list of packages each of them matches all search terms
"""
instance = cls()
packages: Dict[str, AURPackage] = {}
for term in filter(lambda word: len(word) > 3, keywords):
portion = instance.search(term)
packages = {
package.name: package # not mistake to group them by name
for package in portion
if package.name in packages or not packages
}
return list(packages.values())
@classmethod
def search(cls: Type[Remote], *keywords: str) -> List[AURPackage]:
"""
search package in AUR web
Args:
*keywords(str): search terms, e.g. "ahriman", "is", "cool"
Returns:
List[AURPackage]: list of packages which match the criteria
"""
return cls().package_search(*keywords)
def package_info(self, package_name: str) -> AURPackage:
"""
get package info by its name
Args:
package_name(str): package name to search
Returns:
AURPackage: package which match the package name
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def package_search(self, *keywords: str) -> List[AURPackage]:
"""
search package in AUR web
Args:
*keywords(str): keywords to search
Returns:
List[AURPackage]: list of packages which match the criteria
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError

View File

@ -30,11 +30,13 @@ from ahriman.models.repository_paths import RepositoryPaths
class Repo: class Repo:
""" """
repo-add and repo-remove wrapper repo-add and repo-remove wrapper
:ivar logger: class logger
:ivar name: repository name Attributes:
:ivar paths: repository paths instance logger(logging.Logger): class logger
:ivar sign_args: additional args which have to be used to sign repository archive name(str): repository name
:ivar uid: uid of the repository owner user paths(RepositoryPaths): repository paths instance
sign_args(List[str]): additional args which have to be used to sign repository archive
uid(int): uid of the repository owner user
""" """
_check_output = check_output _check_output = check_output
@ -42,9 +44,11 @@ class Repo:
def __init__(self, name: str, paths: RepositoryPaths, sign_args: List[str]) -> None: def __init__(self, name: str, paths: RepositoryPaths, sign_args: List[str]) -> None:
""" """
default constructor default constructor
:param name: repository name
:param paths: repository paths instance Args:
:param sign_args: additional args which have to be used to sign repository archive name(str): repository name
paths(RepositoryPaths): repository paths instance
sign_args(List[str]): additional args which have to be used to sign repository archive
""" """
self.logger = logging.getLogger("build_details") self.logger = logging.getLogger("build_details")
self.name = name self.name = name
@ -55,14 +59,17 @@ class Repo:
@property @property
def repo_path(self) -> Path: def repo_path(self) -> Path:
""" """
:return: path to repository database Returns:
Path: path to repository database
""" """
return self.paths.repository / f"{self.name}.db.tar.gz" return self.paths.repository / f"{self.name}.db.tar.gz"
def add(self, path: Path) -> None: def add(self, path: Path) -> None:
""" """
add new package to repository add new package to repository
:param path: path to archive to add
Args:
path(Path): path to archive to add
""" """
Repo._check_output( Repo._check_output(
"repo-add", *self.sign_args, "-R", str(self.repo_path), str(path), "repo-add", *self.sign_args, "-R", str(self.repo_path), str(path),
@ -85,8 +92,10 @@ class Repo:
def remove(self, package: str, filename: Path) -> None: def remove(self, package: str, filename: Path) -> None:
""" """
remove package from repository remove package from repository
:param package: package name to remove
:param filename: package filename to remove Args:
package(str): package name to remove
filename(Path): package filename to remove
""" """
# remove package and signature (if any) from filesystem # remove package and signature (if any) from filesystem
for full_path in self.paths.repository.glob(f"{filename}*"): for full_path in self.paths.repository.glob(f"{filename}*"):

View File

@ -32,16 +32,21 @@ from ahriman.models.user_access import UserAccess
class Auth: class Auth:
""" """
helper to deal with user authorization helper to deal with user authorization
:ivar enabled: indicates if authorization is enabled
:ivar max_age: session age in seconds. It will be used for both client side and server side checks Attributes:
:ivar safe_build_status: allow read only access to the index page enabled(bool): indicates if authorization is enabled
logger(logging.Logger): class logger
max_age(int): session age in seconds. It will be used for both client side and server side checks
safe_build_status(bool): allow read only access to the index page
""" """
def __init__(self, configuration: Configuration, provider: AuthSettings = AuthSettings.Disabled) -> None: def __init__(self, configuration: Configuration, provider: AuthSettings = AuthSettings.Disabled) -> None:
""" """
default constructor default constructor
:param configuration: configuration instance
:param provider: authorization type definition Args:
configuration(Configuration): configuration instance
provider(AuthSettings, optional): authorization type definition (Default value = AuthSettings.Disabled)
""" """
self.logger = logging.getLogger("http") self.logger = logging.getLogger("http")
@ -57,7 +62,9 @@ class Auth:
In case of internal authentication it must provide an interface (modal form) to login with button sends POST In case of internal authentication it must provide an interface (modal form) to login with button sends POST
request. But for an external providers behaviour can be different: e.g. OAuth provider requires sending GET request. But for an external providers behaviour can be different: e.g. OAuth provider requires sending GET
request to external resource request to external resource
:return: login control as html code to insert
Returns:
str: login control as html code to insert
""" """
return """<button type="button" class="btn btn-link" data-bs-toggle="modal" data-bs-target="#loginForm" style="text-decoration: none">login</button>""" return """<button type="button" class="btn btn-link" data-bs-toggle="modal" data-bs-target="#loginForm" style="text-decoration: none">login</button>"""
@ -65,9 +72,13 @@ class Auth:
def load(cls: Type[Auth], configuration: Configuration, database: SQLite) -> Auth: def load(cls: Type[Auth], configuration: Configuration, database: SQLite) -> Auth:
""" """
load authorization module from settings load authorization module from settings
:param configuration: configuration instance
:param database: database instance Args:
:return: authorization module according to current settings configuration(Configuration): configuration instance
database(SQLite): database instance
Returns:
Auth: authorization module according to current settings
""" """
provider = AuthSettings.from_option(configuration.get("auth", "target", fallback="disabled")) provider = AuthSettings.from_option(configuration.get("auth", "target", fallback="disabled"))
if provider == AuthSettings.Configuration: if provider == AuthSettings.Configuration:
@ -81,9 +92,13 @@ class Auth:
async def check_credentials(self, username: Optional[str], password: Optional[str]) -> bool: # pylint: disable=no-self-use async def check_credentials(self, username: Optional[str], password: Optional[str]) -> bool: # pylint: disable=no-self-use
""" """
validate user password validate user password
:param username: username
:param password: entered password Args:
:return: True in case if password matches, False otherwise username(Optional[str]): username
password(Optional[str]): entered password
Returns:
bool: True in case if password matches, False otherwise
""" """
del username, password del username, password
return True return True
@ -91,8 +106,12 @@ class Auth:
async def known_username(self, username: Optional[str]) -> bool: # pylint: disable=no-self-use async def known_username(self, username: Optional[str]) -> bool: # pylint: disable=no-self-use
""" """
check if user is known check if user is known
:param username: username
:return: True in case if user is known and can be authorized and False otherwise Args:
username(Optional[str]): username
Returns:
bool: True in case if user is known and can be authorized and False otherwise
""" """
del username del username
return True return True
@ -100,10 +119,14 @@ class Auth:
async def verify_access(self, username: str, required: UserAccess, context: Optional[str]) -> bool: # pylint: disable=no-self-use async def verify_access(self, username: str, required: UserAccess, context: Optional[str]) -> bool: # pylint: disable=no-self-use
""" """
validate if user has access to requested resource validate if user has access to requested resource
:param username: username
:param required: required access level Args:
:param context: URI request path username(str): username
:return: True in case if user is allowed to do this request and False otherwise required(UserAccess): required access level
context(Optional[str]): URI request path
Returns:
bool: True in case if user is allowed to do this request and False otherwise
""" """
del username, required, context del username, required, context
return True return True

View File

@ -29,8 +29,12 @@ except ImportError:
async def authorized_userid(*args: Any) -> Any: async def authorized_userid(*args: Any) -> Any:
""" """
handle aiohttp security methods handle aiohttp security methods
:param args: argument list as provided by authorized_userid function
:return: None in case if no aiohttp_security module found and function call otherwise Args:
*args(Any): argument list as provided by authorized_userid function
Returns:
Any: None in case if no aiohttp_security module found and function call otherwise
""" """
if _has_aiohttp_security: if _has_aiohttp_security:
return await aiohttp_security.authorized_userid(*args) # pylint: disable=no-value-for-parameter return await aiohttp_security.authorized_userid(*args) # pylint: disable=no-value-for-parameter
@ -40,8 +44,12 @@ async def authorized_userid(*args: Any) -> Any:
async def check_authorized(*args: Any) -> Any: async def check_authorized(*args: Any) -> Any:
""" """
handle aiohttp security methods handle aiohttp security methods
:param args: argument list as provided by check_authorized function
:return: None in case if no aiohttp_security module found and function call otherwise Args:
*args(Any): argument list as provided by check_authorized function
Returns:
Any: None in case if no aiohttp_security module found and function call otherwise
""" """
if _has_aiohttp_security: if _has_aiohttp_security:
return await aiohttp_security.check_authorized(*args) # pylint: disable=no-value-for-parameter return await aiohttp_security.check_authorized(*args) # pylint: disable=no-value-for-parameter
@ -51,8 +59,12 @@ async def check_authorized(*args: Any) -> Any:
async def forget(*args: Any) -> Any: async def forget(*args: Any) -> Any:
""" """
handle aiohttp security methods handle aiohttp security methods
:param args: argument list as provided by forget function
:return: None in case if no aiohttp_security module found and function call otherwise Args:
*args(Any): argument list as provided by forget function
Returns:
Any: None in case if no aiohttp_security module found and function call otherwise
""" """
if _has_aiohttp_security: if _has_aiohttp_security:
return await aiohttp_security.forget(*args) # pylint: disable=no-value-for-parameter return await aiohttp_security.forget(*args) # pylint: disable=no-value-for-parameter
@ -62,8 +74,12 @@ async def forget(*args: Any) -> Any:
async def remember(*args: Any) -> Any: async def remember(*args: Any) -> Any:
""" """
handle disabled auth handle disabled auth
:param args: argument list as provided by remember function
:return: None in case if no aiohttp_security module found and function call otherwise Args:
*args(Any): argument list as provided by remember function
Returns:
Any: None in case if no aiohttp_security module found and function call otherwise
""" """
if _has_aiohttp_security: if _has_aiohttp_security:
return await aiohttp_security.remember(*args) # pylint: disable=no-value-for-parameter return await aiohttp_security.remember(*args) # pylint: disable=no-value-for-parameter

View File

@ -31,17 +31,21 @@ from ahriman.models.user_access import UserAccess
class Mapping(Auth): class Mapping(Auth):
""" """
user authorization based on mapping from configuration file user authorization based on mapping from configuration file
:ivar salt: random generated string to salt passwords
:ivar database: database instance Attributes:
salt(str): random generated string to salt passwords
database(SQLite): database instance
""" """
def __init__(self, configuration: Configuration, database: SQLite, def __init__(self, configuration: Configuration, database: SQLite,
provider: AuthSettings = AuthSettings.Configuration) -> None: provider: AuthSettings = AuthSettings.Configuration) -> None:
""" """
default constructor default constructor
:param configuration: configuration instance
:param database: database instance Args:
:param provider: authorization type definition configuration(Configuration): configuration instance
database(SQLite): database instance
provider(AuthSettings, optional): authorization type definition (Default value = AuthSettings.Configuration)
""" """
Auth.__init__(self, configuration, provider) Auth.__init__(self, configuration, provider)
self.database = database self.database = database
@ -50,9 +54,13 @@ class Mapping(Auth):
async def check_credentials(self, username: Optional[str], password: Optional[str]) -> bool: async def check_credentials(self, username: Optional[str], password: Optional[str]) -> bool:
""" """
validate user password validate user password
:param username: username
:param password: entered password Args:
:return: True in case if password matches, False otherwise username(Optional[str]): username
password(Optional[str]): entered password
Returns:
bool: True in case if password matches, False otherwise
""" """
if username is None or password is None: if username is None or password is None:
return False # invalid data supplied return False # invalid data supplied
@ -62,26 +70,38 @@ class Mapping(Auth):
def get_user(self, username: str) -> Optional[User]: def get_user(self, username: str) -> Optional[User]:
""" """
retrieve user from in-memory mapping retrieve user from in-memory mapping
:param username: username
:return: user descriptor if username is known and None otherwise Args:
username(str): username
Returns:
Optional[User]: user descriptor if username is known and None otherwise
""" """
return self.database.user_get(username) return self.database.user_get(username)
async def known_username(self, username: Optional[str]) -> bool: async def known_username(self, username: Optional[str]) -> bool:
""" """
check if user is known check if user is known
:param username: username
:return: True in case if user is known and can be authorized and False otherwise Args:
username(Optional[str]): username
Returns:
bool: True in case if user is known and can be authorized and False otherwise
""" """
return username is not None and self.get_user(username) is not None return username is not None and self.get_user(username) is not None
async def verify_access(self, username: str, required: UserAccess, context: Optional[str]) -> bool: async def verify_access(self, username: str, required: UserAccess, context: Optional[str]) -> bool:
""" """
validate if user has access to requested resource validate if user has access to requested resource
:param username: username
:param required: required access level Args:
:param context: URI request path username(str): username
:return: True in case if user is allowed to do this request and False otherwise required(UserAccess): required access level
context(Optional[str]): URI request path
Returns:
bool: True in case if user is allowed to do this request and False otherwise
""" """
user = self.get_user(username) user = self.get_user(username)
return user is not None and user.verify_access(required) return user is not None and user.verify_access(required)

View File

@ -32,20 +32,24 @@ class OAuth(Mapping):
""" """
OAuth user authorization. OAuth user authorization.
It is required to create application first and put application credentials. It is required to create application first and put application credentials.
:ivar client_id: application client id
:ivar client_secret: application client secret key Attributes:
:ivar provider: provider class, should be one of aiohttp-client provided classes client_id(str): application client id
:ivar redirect_uri: redirect URI registered in provider client_secret(str): application client secret key
:ivar scopes: list of scopes required by the application provider(aioauth_client.OAuth2Client): provider class, should be one of aiohttp-client provided classes
redirect_uri(str): redirect URI registered in provider
scopes(str): list of scopes required by the application
""" """
def __init__(self, configuration: Configuration, database: SQLite, def __init__(self, configuration: Configuration, database: SQLite,
provider: AuthSettings = AuthSettings.OAuth) -> None: provider: AuthSettings = AuthSettings.OAuth) -> None:
""" """
default constructor default constructor
:param configuration: configuration instance
:param database: database instance Args:
:param provider: authorization type definition configuration(Configuration): configuration instance
database(SQLite): database instance
provider(AuthSettings, optional): authorization type definition (Default value = AuthSettings.OAuth)
""" """
Mapping.__init__(self, configuration, database, provider) Mapping.__init__(self, configuration, database, provider)
self.client_id = configuration.get("auth", "client_id") self.client_id = configuration.get("auth", "client_id")
@ -60,7 +64,8 @@ class OAuth(Mapping):
@property @property
def auth_control(self) -> str: def auth_control(self) -> str:
""" """
:return: login control as html code to insert Returns:
str: login control as html code to insert
""" """
return """<a class="nav-link" href="/user-api/v1/login" title="login via OAuth2">login</a>""" return """<a class="nav-link" href="/user-api/v1/login" title="login via OAuth2">login</a>"""
@ -68,8 +73,15 @@ class OAuth(Mapping):
def get_provider(name: str) -> Type[aioauth_client.OAuth2Client]: def get_provider(name: str) -> Type[aioauth_client.OAuth2Client]:
""" """
load OAuth2 provider by name load OAuth2 provider by name
:param name: name of the provider. Must be valid class defined in aioauth-client library
:return: loaded provider type Args:
name(str): name of the provider. Must be valid class defined in aioauth-client library
Returns:
Type[aioauth_client.OAuth2Client]: loaded provider type
Raises:
InvalidOption: in case if invalid OAuth provider name supplied
""" """
provider: Type[aioauth_client.OAuth2Client] = getattr(aioauth_client, name) provider: Type[aioauth_client.OAuth2Client] = getattr(aioauth_client, name)
try: try:
@ -83,14 +95,18 @@ class OAuth(Mapping):
def get_client(self) -> aioauth_client.OAuth2Client: def get_client(self) -> aioauth_client.OAuth2Client:
""" """
load client from parameters load client from parameters
:return: generated client according to current settings
Returns:
aioauth_client.OAuth2Client: generated client according to current settings
""" """
return self.provider(client_id=self.client_id, client_secret=self.client_secret) return self.provider(client_id=self.client_id, client_secret=self.client_secret)
def get_oauth_url(self) -> str: def get_oauth_url(self) -> str:
""" """
get authorization URI for the specified settings get authorization URI for the specified settings
:return: authorization URI as a string
Returns:
str: authorization URI as a string
""" """
client = self.get_client() client = self.get_client()
uri: str = client.get_authorize_url(scope=self.scopes, redirect_uri=self.redirect_uri) uri: str = client.get_authorize_url(scope=self.scopes, redirect_uri=self.redirect_uri)
@ -99,8 +115,12 @@ class OAuth(Mapping):
async def get_oauth_username(self, code: str) -> Optional[str]: async def get_oauth_username(self, code: str) -> Optional[str]:
""" """
extract OAuth username from remote extract OAuth username from remote
:param code: authorization code provided by external service
:return: username as is in OAuth provider Args:
code(str): authorization code provided by external service
Returns:
Optional[str]: username as is in OAuth provider
""" """
try: try:
client = self.get_client() client = self.get_client()

View File

@ -28,7 +28,9 @@ from ahriman.core.util import check_output
class Sources: class Sources:
""" """
helper to download package sources (PKGBUILD etc) helper to download package sources (PKGBUILD etc)
:cvar logger: class logger
Attributes:
logger(logging.Logger): (class attribute) class logger
""" """
logger = logging.getLogger("build_details") logger = logging.getLogger("build_details")
@ -40,8 +42,10 @@ class Sources:
def add(sources_dir: Path, *pattern: str) -> None: def add(sources_dir: Path, *pattern: str) -> None:
""" """
track found files via git track found files via git
:param sources_dir: local path to git repository
:param pattern: glob patterns Args:
sources_dir(Path): local path to git repository
*pattern(str): glob patterns
""" """
# glob directory to find files which match the specified patterns # glob directory to find files which match the specified patterns
found_files: List[Path] = [] found_files: List[Path] = []
@ -59,8 +63,12 @@ class Sources:
def diff(sources_dir: Path) -> str: def diff(sources_dir: Path) -> str:
""" """
generate diff from the current version and write it to the output file generate diff from the current version and write it to the output file
:param sources_dir: local path to git repository
:return: patch as plain string Args:
sources_dir(Path): local path to git repository
Returns:
str: patch as plain string
""" """
return Sources._check_output("git", "diff", exception=None, cwd=sources_dir, logger=Sources.logger) return Sources._check_output("git", "diff", exception=None, cwd=sources_dir, logger=Sources.logger)
@ -68,8 +76,10 @@ class Sources:
def fetch(sources_dir: Path, remote: Optional[str]) -> None: def fetch(sources_dir: Path, remote: Optional[str]) -> None:
""" """
either clone repository or update it to origin/`branch` either clone repository or update it to origin/`branch`
:param sources_dir: local path to fetch
:param remote: remote target (from where to fetch) Args:
sources_dir(Path): local path to fetch
remote(Optional[str]): remote target (from where to fetch)
""" """
# local directory exists and there is .git directory # local directory exists and there is .git directory
is_initialized_git = (sources_dir / ".git").is_dir() is_initialized_git = (sources_dir / ".git").is_dir()
@ -86,7 +96,8 @@ class Sources:
Sources.logger.warning("%s is not initialized, but no remote provided", sources_dir) Sources.logger.warning("%s is not initialized, but no remote provided", sources_dir)
else: else:
Sources.logger.info("clone remote %s to %s", remote, sources_dir) Sources.logger.info("clone remote %s to %s", remote, sources_dir)
Sources._check_output("git", "clone", remote, str(sources_dir), exception=None, logger=Sources.logger) Sources._check_output("git", "clone", remote, str(sources_dir),
exception=None, cwd=sources_dir, logger=Sources.logger)
# and now force reset to our branch # and now force reset to our branch
Sources._check_output("git", "checkout", "--force", Sources._branch, Sources._check_output("git", "checkout", "--force", Sources._branch,
exception=None, cwd=sources_dir, logger=Sources.logger) exception=None, cwd=sources_dir, logger=Sources.logger)
@ -97,8 +108,12 @@ class Sources:
def has_remotes(sources_dir: Path) -> bool: def has_remotes(sources_dir: Path) -> bool:
""" """
check if there are remotes for the repository check if there are remotes for the repository
:param sources_dir: local path to git repository
:return: True in case if there is any remote and false otherwise Args:
sources_dir(Path): local path to git repository
Returns:
bool: True in case if there is any remote and false otherwise
""" """
remotes = Sources._check_output("git", "remote", exception=None, cwd=sources_dir, logger=Sources.logger) remotes = Sources._check_output("git", "remote", exception=None, cwd=sources_dir, logger=Sources.logger)
return bool(remotes) return bool(remotes)
@ -107,7 +122,9 @@ class Sources:
def init(sources_dir: Path) -> None: def init(sources_dir: Path) -> None:
""" """
create empty git repository at the specified path create empty git repository at the specified path
:param sources_dir: local path to sources
Args:
sources_dir(Path): local path to sources
""" """
Sources._check_output("git", "init", "--initial-branch", Sources._branch, Sources._check_output("git", "init", "--initial-branch", Sources._branch,
exception=None, cwd=sources_dir, logger=Sources.logger) exception=None, cwd=sources_dir, logger=Sources.logger)
@ -116,9 +133,11 @@ class Sources:
def load(sources_dir: Path, remote: str, patch: Optional[str]) -> None: def load(sources_dir: Path, remote: str, patch: Optional[str]) -> None:
""" """
fetch sources from remote and apply patches fetch sources from remote and apply patches
:param sources_dir: local path to fetch
:param remote: remote target (from where to fetch) Args:
:param patch: optional patch to be applied sources_dir(Path): local path to fetch
remote(str): remote target (from where to fetch)
patch(Optional[str]): optional patch to be applied
""" """
Sources.fetch(sources_dir, remote) Sources.fetch(sources_dir, remote)
if patch is None: if patch is None:
@ -130,8 +149,10 @@ class Sources:
def patch_apply(sources_dir: Path, patch: str) -> None: def patch_apply(sources_dir: Path, patch: str) -> None:
""" """
apply patches if any apply patches if any
:param sources_dir: local path to directory with git sources
:param patch: patch to be applied Args:
sources_dir(Path): local path to directory with git sources
patch(str): patch to be applied
""" """
# create patch # create patch
Sources.logger.info("apply patch from database") Sources.logger.info("apply patch from database")
@ -142,9 +163,13 @@ class Sources:
def patch_create(sources_dir: Path, *pattern: str) -> str: def patch_create(sources_dir: Path, *pattern: str) -> str:
""" """
create patch set for the specified local path create patch set for the specified local path
:param sources_dir: local path to git repository
:param pattern: glob patterns Args:
:return: patch as plain text sources_dir(Path): local path to git repository
*pattern(str): glob patterns
Returns:
str: patch as plain text
""" """
Sources.add(sources_dir, *pattern) Sources.add(sources_dir, *pattern)
diff = Sources.diff(sources_dir) diff = Sources.diff(sources_dir)

View File

@ -35,11 +35,13 @@ from ahriman.models.repository_paths import RepositoryPaths
class Task: class Task:
""" """
base package build task base package build task
:ivar build_logger: logger for build process
:ivar logger: class logger Attributes:
:ivar package: package definitions build_logger(logging.Logger): logger for build process
:ivar paths: repository paths instance logger(logging.Logger): class logger
:ivar uid: uid of the repository owner user package(Package): package definitions
paths(RepositoryPaths): repository paths instance
uid(int): uid of the repository owner user
""" """
_check_output = check_output _check_output = check_output
@ -47,9 +49,11 @@ class Task:
def __init__(self, package: Package, configuration: Configuration, paths: RepositoryPaths) -> None: def __init__(self, package: Package, configuration: Configuration, paths: RepositoryPaths) -> None:
""" """
default constructor default constructor
:param package: package definitions
:param configuration: configuration instance Args:
:param paths: repository paths instance package(Package): package definitions
configuration(Configuration): configuration instance
paths(RepositoryPaths): repository paths instance
""" """
self.logger = logging.getLogger("root") self.logger = logging.getLogger("root")
self.build_logger = logging.getLogger("build_details") self.build_logger = logging.getLogger("build_details")
@ -65,8 +69,12 @@ class Task:
def build(self, sources_path: Path) -> List[Path]: def build(self, sources_path: Path) -> List[Path]:
""" """
run package build run package build
:param sources_path: path to where sources are
:return: paths of produced packages Args:
sources_path(Path): path to where sources are
Returns:
List[Path]: paths of produced packages
""" """
command = [self.build_command, "-r", str(self.paths.chroot)] command = [self.build_command, "-r", str(self.paths.chroot)]
command.extend(self.archbuild_flags) command.extend(self.archbuild_flags)
@ -91,8 +99,10 @@ class Task:
def init(self, path: Path, database: SQLite) -> None: def init(self, path: Path, database: SQLite) -> None:
""" """
fetch package from git fetch package from git
:param path: local path to fetch
:param database: database instance Args:
path(Path): local path to fetch
database(SQLite): database instance
""" """
if self.paths.cache_for(self.package.base).is_dir(): if self.paths.cache_for(self.package.base).is_dir():
# no need to clone whole repository, just copy from cache first # no need to clone whole repository, just copy from cache first

View File

@ -34,12 +34,15 @@ from ahriman.models.repository_paths import RepositoryPaths
class Configuration(configparser.RawConfigParser): class Configuration(configparser.RawConfigParser):
""" """
extension for built-in configuration parser extension for built-in configuration parser
:ivar architecture: repository architecture
:ivar path: path to root configuration file Attributes:
:cvar ARCHITECTURE_SPECIFIC_SECTIONS: known sections which can be architecture specific (required by dump) ARCHITECTURE_SPECIFIC_SECTIONS(List[str]): (class attribute) known sections which can be architecture specific.
:cvar DEFAULT_LOG_FORMAT: default log format (in case of fallback) Required by dump and merging functions
:cvar DEFAULT_LOG_LEVEL: default log level (in case of fallback) DEFAULT_LOG_FORMAT(str): (class attribute) default log format (in case of fallback)
:cvar SYSTEM_CONFIGURATION_PATH: default system configuration path distributed by package DEFAULT_LOG_LEVEL(int): (class attribute) default log level (in case of fallback)
SYSTEM_CONFIGURATION_PATH(Path): (class attribute) default system configuration path distributed by package
architecture(Optional[str]): repository architecture
path(Optional[Path]): path to root configuration file
""" """
DEFAULT_LOG_FORMAT = "[%(levelname)s %(asctime)s] [%(filename)s:%(lineno)d %(funcName)s]: %(message)s" DEFAULT_LOG_FORMAT = "[%(levelname)s %(asctime)s] [%(filename)s:%(lineno)d %(funcName)s]: %(message)s"
@ -62,21 +65,24 @@ class Configuration(configparser.RawConfigParser):
@property @property
def include(self) -> Path: def include(self) -> Path:
""" """
:return: path to directory with configuration includes Returns:
Path: path to directory with configuration includes
""" """
return self.getpath("settings", "include") return self.getpath("settings", "include")
@property @property
def logging_path(self) -> Path: def logging_path(self) -> Path:
""" """
:return: path to logging configuration Returns:
Path: path to logging configuration
""" """
return self.getpath("settings", "logging") return self.getpath("settings", "logging")
@property @property
def repository_paths(self) -> RepositoryPaths: def repository_paths(self) -> RepositoryPaths:
""" """
:return: repository paths instance Returns:
RepositoryPaths: repository paths instance
""" """
_, architecture = self.check_loaded() _, architecture = self.check_loaded()
return RepositoryPaths(self.getpath("repository", "root"), architecture) return RepositoryPaths(self.getpath("repository", "root"), architecture)
@ -85,10 +91,14 @@ class Configuration(configparser.RawConfigParser):
def from_path(cls: Type[Configuration], path: Path, architecture: str, quiet: bool) -> Configuration: def from_path(cls: Type[Configuration], path: Path, architecture: str, quiet: bool) -> Configuration:
""" """
constructor with full object initialization constructor with full object initialization
:param path: path to root configuration file
:param architecture: repository architecture Args:
:param quiet: force disable any log messages path(Path): path to root configuration file
:return: configuration instance architecture(str): repository architecture
quiet(bool): force disable any log messages
Returns:
Configuration: configuration instance
""" """
config = cls() config = cls()
config.load(path) config.load(path)
@ -100,8 +110,15 @@ class Configuration(configparser.RawConfigParser):
def __convert_list(value: str) -> List[str]: def __convert_list(value: str) -> List[str]:
""" """
convert string value to list of strings convert string value to list of strings
:param value: string configuration value
:return: list of string from the parsed string Args:
value(str): string configuration value
Returns:
List[str]: list of string from the parsed string
Raises:
ValueError: in case if option value contains unclosed quotes
""" """
def generator() -> Generator[str, None, None]: def generator() -> Generator[str, None, None]:
quote_mark = None quote_mark = None
@ -111,7 +128,7 @@ class Configuration(configparser.RawConfigParser):
quote_mark = char quote_mark = char
elif char == quote_mark: # quoted part ended, reset quotation elif char == quote_mark: # quoted part ended, reset quotation
quote_mark = None quote_mark = None
elif char == " " and quote_mark is None: # found space outside of the quotation, yield the word elif char == " " and quote_mark is None: # found space outside the quotation, yield the word
yield word yield word
word = "" word = ""
else: # append character to the buffer else: # append character to the buffer
@ -126,17 +143,25 @@ class Configuration(configparser.RawConfigParser):
def section_name(section: str, suffix: str) -> str: def section_name(section: str, suffix: str) -> str:
""" """
generate section name for sections which depends on context generate section name for sections which depends on context
:param section: section name
:param suffix: session suffix, e.g. repository architecture Args:
:return: correct section name for repository specific section section(str): section name
suffix(str): session suffix, e.g. repository architecture
Returns:
str: correct section name for repository specific section
""" """
return f"{section}:{suffix}" return f"{section}:{suffix}"
def __convert_path(self, value: str) -> Path: def __convert_path(self, value: str) -> Path:
""" """
convert string value to path object convert string value to path object
:param value: string configuration value
:return: path object which represents the configuration value Args:
value(str): string configuration value
Returns:
Path: path object which represents the configuration value
""" """
path = Path(value) path = Path(value)
if self.path is None or path.is_absolute(): if self.path is None or path.is_absolute():
@ -146,7 +171,12 @@ class Configuration(configparser.RawConfigParser):
def check_loaded(self) -> Tuple[Path, str]: def check_loaded(self) -> Tuple[Path, str]:
""" """
check if service was actually loaded check if service was actually loaded
:return: configuration root path and architecture if loaded
Returns:
Tuple[Path, str]: configuration root path and architecture if loaded
Raises:
InitializeException: in case if architecture and/or path are not set
""" """
if self.path is None or self.architecture is None: if self.path is None or self.architecture is None:
raise InitializeException("Configuration path and/or architecture are not set") raise InitializeException("Configuration path and/or architecture are not set")
@ -155,7 +185,9 @@ class Configuration(configparser.RawConfigParser):
def dump(self) -> Dict[str, Dict[str, str]]: def dump(self) -> Dict[str, Dict[str, str]]:
""" """
dump configuration to dictionary dump configuration to dictionary
:return: configuration dump for specific architecture
Returns:
Dict[str, Dict[str, str]]: configuration dump for specific architecture
""" """
return { return {
section: dict(self[section]) section: dict(self[section])
@ -172,9 +204,16 @@ class Configuration(configparser.RawConfigParser):
""" """
get type variable with fallback to old logic get type variable with fallback to old logic
Despite the fact that it has same semantics as other get* methods, but it has different argument list Despite the fact that it has same semantics as other get* methods, but it has different argument list
:param section: section name
:param architecture: repository architecture Args:
:return: section name and found type name section(str): section name
architecture(str): repository architecture
Returns:
Tuple[str, str]: section name and found type name
Raises:
configparser.NoSectionError: in case if no section found
""" """
group_type = self.get(section, "type", fallback=None) # new-style logic group_type = self.get(section, "type", fallback=None) # new-style logic
if group_type is not None: if group_type is not None:
@ -191,7 +230,9 @@ class Configuration(configparser.RawConfigParser):
def load(self, path: Path) -> None: def load(self, path: Path) -> None:
""" """
fully load configuration fully load configuration
:param path: path to root configuration file
Args:
path(Path): path to root configuration file
""" """
if not path.is_file(): # fallback to the system file if not path.is_file(): # fallback to the system file
path = self.SYSTEM_CONFIGURATION_PATH path = self.SYSTEM_CONFIGURATION_PATH
@ -214,7 +255,9 @@ class Configuration(configparser.RawConfigParser):
def load_logging(self, quiet: bool) -> None: def load_logging(self, quiet: bool) -> None:
""" """
setup logging settings from configuration setup logging settings from configuration
:param quiet: force disable any log messages
Args:
quiet(bool): force disable any log messages
""" """
try: try:
path = self.logging_path path = self.logging_path
@ -229,7 +272,9 @@ class Configuration(configparser.RawConfigParser):
def merge_sections(self, architecture: str) -> None: def merge_sections(self, architecture: str) -> None:
""" """
merge architecture specific sections into main configuration merge architecture specific sections into main configuration
:param architecture: repository architecture
Args:
architecture(str): repository architecture
""" """
self.architecture = architecture self.architecture = architecture
for section in self.ARCHITECTURE_SPECIFIC_SECTIONS: for section in self.ARCHITECTURE_SPECIFIC_SECTIONS:
@ -260,9 +305,11 @@ class Configuration(configparser.RawConfigParser):
def set_option(self, section: str, option: str, value: Optional[str]) -> None: def set_option(self, section: str, option: str, value: Optional[str]) -> None:
""" """
set option. Unlike default `configparser.RawConfigParser.set` it also creates section if it does not exist set option. Unlike default `configparser.RawConfigParser.set` it also creates section if it does not exist
:param section: section name
:param option: option name Args:
:param value: option value as string in parsable format section(str): section name
option(str): option name
value(Optional[str]): option value as string in parsable format
""" """
if not self.has_section(section): if not self.has_section(section):
self.add_section(section) self.add_section(section)

View File

@ -20,9 +20,9 @@
from sqlite3 import Connection from sqlite3 import Connection
from ahriman.core.configuration import Configuration from ahriman.core.configuration import Configuration
from ahriman.core.database.data.package_statuses import migrate_package_statuses
from ahriman.core.database.data.patches import migrate_patches from ahriman.core.database.data.patches import migrate_patches
from ahriman.core.database.data.users import migrate_users_data from ahriman.core.database.data.users import migrate_users_data
from ahriman.core.database.data.package_statuses import migrate_package_statuses
from ahriman.models.migration_result import MigrationResult from ahriman.models.migration_result import MigrationResult
from ahriman.models.repository_paths import RepositoryPaths from ahriman.models.repository_paths import RepositoryPaths
@ -31,13 +31,15 @@ def migrate_data(result: MigrationResult, connection: Connection,
configuration: Configuration, paths: RepositoryPaths) -> None: configuration: Configuration, paths: RepositoryPaths) -> None:
""" """
perform data migration perform data migration
:param result: result of the schema migration
:param connection: database connection Args:
:param configuration: configuration instance result(MigrationResult): result of the schema migration
:param paths: repository paths instance connection(Connection): database connection
configuration(Configuration): configuration instance
paths(RepositoryPaths): repository paths instance
""" """
# initial data migration # initial data migration
if result.old_version == 0: if result.old_version <= 0:
migrate_package_statuses(connection, paths) migrate_package_statuses(connection, paths)
migrate_users_data(connection, configuration)
migrate_patches(connection, paths) migrate_patches(connection, paths)
migrate_users_data(connection, configuration)

View File

@ -29,8 +29,10 @@ from ahriman.models.repository_paths import RepositoryPaths
def migrate_package_statuses(connection: Connection, paths: RepositoryPaths) -> None: def migrate_package_statuses(connection: Connection, paths: RepositoryPaths) -> None:
""" """
perform migration for package statuses perform migration for package statuses
:param connection: database connection
:param paths: repository paths instance Args:
connection(Connection): database connection
paths(RepositoryPaths): repository paths instance
""" """
def insert_base(metadata: Package, last_status: BuildStatus) -> None: def insert_base(metadata: Package, last_status: BuildStatus) -> None:
connection.execute( connection.execute(

View File

@ -25,8 +25,10 @@ from ahriman.models.repository_paths import RepositoryPaths
def migrate_patches(connection: Connection, paths: RepositoryPaths) -> None: def migrate_patches(connection: Connection, paths: RepositoryPaths) -> None:
""" """
perform migration for patches perform migration for patches
:param connection: database connection
:param paths: repository paths instance Args:
connection(Connection): database connection
paths(RepositoryPaths): repository paths instance
""" """
root = paths.root / "patches" root = paths.root / "patches"
if not root.is_dir(): if not root.is_dir():

View File

@ -25,16 +25,18 @@ from ahriman.core.configuration import Configuration
def migrate_users_data(connection: Connection, configuration: Configuration) -> None: def migrate_users_data(connection: Connection, configuration: Configuration) -> None:
""" """
perform migration for users perform migration for users
:param connection: database connection
:param configuration: configuration instance Args:
connection(Connection): database connection
configuration(Configuration): configuration instance
""" """
for section in configuration.sections(): for section in configuration.sections():
for option, value in configuration[section].items(): for option, value in configuration[section].items():
if not section.startswith("auth:"): if not section.startswith("auth:"):
continue continue
permission = section[5:] access = section[5:]
connection.execute( connection.execute(
"""insert into users (username, permission, password) values (:username, :permission, :password)""", """insert into users (username, access, password) values (:username, :access, :password)""",
{"username": option.lower(), "permission": permission, "password": value}) {"username": option.lower(), "access": access, "password": value})
connection.commit() connection.commit()

View File

@ -35,14 +35,18 @@ class Migrations:
""" """
simple migration wrapper for the sqlite simple migration wrapper for the sqlite
idea comes from https://www.ash.dev/blog/simple-migration-system-in-sqlite/ idea comes from https://www.ash.dev/blog/simple-migration-system-in-sqlite/
:ivar connection: database connection
:ivar logger: class logger Attributes:
connection(Connection): database connection
logger(logging.Logger): class logger
""" """
def __init__(self, connection: Connection) -> None: def __init__(self, connection: Connection) -> None:
""" """
default constructor default constructor
:param connection: database connection
Args:
connection(Connection): database connection
""" """
self.connection = connection self.connection = connection
self.logger = logging.getLogger("database") self.logger = logging.getLogger("database")
@ -51,8 +55,12 @@ class Migrations:
def migrate(cls: Type[Migrations], connection: Connection) -> MigrationResult: def migrate(cls: Type[Migrations], connection: Connection) -> MigrationResult:
""" """
perform migrations implicitly perform migrations implicitly
:param connection: database connection
:return: current schema version Args:
connection(Connection): database connection
Returns:
MigrationResult: current schema version
""" """
return cls(connection).run() return cls(connection).run()
@ -61,6 +69,8 @@ class Migrations:
extract all migrations from the current package extract all migrations from the current package
idea comes from https://julienharbulot.com/python-dynamical-import.html idea comes from https://julienharbulot.com/python-dynamical-import.html
Returns:
List[Migration]: list of found migrations
""" """
migrations: List[Migration] = [] migrations: List[Migration] = []
package_dir = Path(__file__).resolve().parent package_dir = Path(__file__).resolve().parent
@ -77,7 +87,9 @@ class Migrations:
def run(self) -> MigrationResult: def run(self) -> MigrationResult:
""" """
perform migrations perform migrations
:return: current schema version
Return:
MigrationResult: current schema version
""" """
migrations = self.migrations() migrations = self.migrations()
current_version = self.user_version() current_version = self.user_version()
@ -118,7 +130,9 @@ class Migrations:
def user_version(self) -> int: def user_version(self) -> int:
""" """
get schema version from sqlite database get schema version from sqlite database
;return: current schema version
Returns:
int: current schema version
""" """
cursor = self.connection.execute("pragma user_version") cursor = self.connection.execute("pragma user_version")
current_version: int = cursor.fetchone()["user_version"] current_version: int = cursor.fetchone()["user_version"]

View File

@ -17,8 +17,6 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
# #
from __future__ import annotations
from sqlite3 import Connection from sqlite3 import Connection
from typing import List, Optional from typing import List, Optional
@ -35,17 +33,25 @@ class AuthOperations(Operations):
def user_get(self, username: str) -> Optional[User]: def user_get(self, username: str) -> Optional[User]:
""" """
get user by username get user by username
:param username: username
:return: user if it was found Args:
username(str): username
Returns:
Optional[User]: user if it was found
""" """
return next(iter(self.user_list(username, None)), None) return next(iter(self.user_list(username, None)), None)
def user_list(self, username: Optional[str], access: Optional[UserAccess]) -> List[User]: def user_list(self, username: Optional[str], access: Optional[UserAccess]) -> List[User]:
""" """
get users by filter get users by filter
:param username: optional filter by username
:param access: optional filter by role Args:
:return: list of users who match criteria username(Optional[str]): optional filter by username
access(Optional[UserAccess]): optional filter by role
Returns:
List[User]: list of users who match criteria
""" """
username_filter = username.lower() if username is not None else username username_filter = username.lower() if username is not None else username
access_filter = access.value if access is not None else access access_filter = access.value if access is not None else access
@ -66,7 +72,9 @@ class AuthOperations(Operations):
def user_remove(self, username: str) -> None: def user_remove(self, username: str) -> None:
""" """
remove user from storage remove user from storage
:param username: username
Args:
username(str): username
""" """
def run(connection: Connection) -> None: def run(connection: Connection) -> None:
connection.execute("""delete from users where username = :username""", {"username": username.lower()}) connection.execute("""delete from users where username = :username""", {"username": username.lower()})
@ -75,8 +83,10 @@ class AuthOperations(Operations):
def user_update(self, user: User) -> None: def user_update(self, user: User) -> None:
""" """
get user by username update user by username
:param user: user descriptor
Args:
user(User): user descriptor
""" """
def run(connection: Connection) -> None: def run(connection: Connection) -> None:
connection.execute( connection.execute(

View File

@ -32,7 +32,9 @@ class BuildOperations(Operations):
def build_queue_clear(self, package_base: Optional[str]) -> None: def build_queue_clear(self, package_base: Optional[str]) -> None:
""" """
remove packages from build queue remove packages from build queue
:param package_base: optional filter by package base
Args:
package_base(Optional[str]): optional filter by package base
""" """
def run(connection: Connection) -> None: def run(connection: Connection) -> None:
connection.execute( connection.execute(
@ -47,7 +49,9 @@ class BuildOperations(Operations):
def build_queue_get(self) -> List[Package]: def build_queue_get(self) -> List[Package]:
""" """
retrieve packages from build queue retrieve packages from build queue
:return: list of packages to be built
Return:
List[Package]: list of packages to be built
""" """
def run(connection: Connection) -> List[Package]: def run(connection: Connection) -> List[Package]:
return [ return [
@ -60,7 +64,9 @@ class BuildOperations(Operations):
def build_queue_insert(self, package: Package) -> None: def build_queue_insert(self, package: Package) -> None:
""" """
insert packages to build queue insert packages to build queue
:param package: package to be inserted
Args:
package(Package): package to be inserted
""" """
def run(connection: Connection) -> None: def run(connection: Connection) -> None:
connection.execute( connection.execute(

View File

@ -31,14 +31,18 @@ T = TypeVar("T")
class Operations: class Operations:
""" """
base operation class base operation class
:ivar logger: class logger
:ivar path: path to the database file Attributes:
logger(logging.Logger): class logger
path(Path): path to the database file
""" """
def __init__(self, path: Path) -> None: def __init__(self, path: Path) -> None:
""" """
default constructor default constructor
:param path: path to the database file
Args:
path(Path): path to the database file
""" """
self.path = path self.path = path
self.logger = logging.getLogger("database") self.logger = logging.getLogger("database")
@ -47,9 +51,13 @@ class Operations:
def factory(cursor: Cursor, row: Tuple[Any, ...]) -> Dict[str, Any]: def factory(cursor: Cursor, row: Tuple[Any, ...]) -> Dict[str, Any]:
""" """
dictionary factory based on official documentation dictionary factory based on official documentation
:param cursor: cursor descriptor
:param row: fetched row Args:
:return: row converted to dictionary cursor(Cursor): cursor descriptor
row(Tuple[Any, ...]): fetched row
Returns:
Dict[str, Any]: row converted to dictionary
""" """
result = {} result = {}
for index, column in enumerate(cursor.description): for index, column in enumerate(cursor.description):
@ -59,9 +67,13 @@ class Operations:
def with_connection(self, query: Callable[[Connection], T], commit: bool = False) -> T: def with_connection(self, query: Callable[[Connection], T], commit: bool = False) -> T:
""" """
perform operation in connection perform operation in connection
:param query: function to be called with connection
:param commit: if True commit() will be called on success Args:
:return: result of the `query` call query(Callable[[Connection], T]): function to be called with connection
commit(bool, optional): if True commit() will be called on success (Default value = False)
Returns:
T: result of the `query` call
""" """
with sqlite3.connect(self.path, detect_types=sqlite3.PARSE_DECLTYPES) as connection: with sqlite3.connect(self.path, detect_types=sqlite3.PARSE_DECLTYPES) as connection:
connection.row_factory = self.factory connection.row_factory = self.factory

View File

@ -35,8 +35,10 @@ class PackageOperations(Operations):
def _package_remove_package_base(connection: Connection, package_base: str) -> None: def _package_remove_package_base(connection: Connection, package_base: str) -> None:
""" """
remove package base information remove package base information
:param connection: database connection
:param package_base: package base name Args:
connection(Connection): database connection
package_base(str): package base name
""" """
connection.execute("""delete from package_statuses where package_base = :package_base""", connection.execute("""delete from package_statuses where package_base = :package_base""",
{"package_base": package_base}) {"package_base": package_base})
@ -47,9 +49,11 @@ class PackageOperations(Operations):
def _package_remove_packages(connection: Connection, package_base: str, current_packages: Iterable[str]) -> None: def _package_remove_packages(connection: Connection, package_base: str, current_packages: Iterable[str]) -> None:
""" """
remove packages belong to the package base remove packages belong to the package base
:param connection: database connection
:param package_base: package base name Args:
:param current_packages: current packages list which has to be left in database connection(Connection): database connection
package_base(str): package base name
current_packages(Iterable[str]): current packages list which has to be left in database
""" """
packages = [ packages = [
package package
@ -63,8 +67,10 @@ class PackageOperations(Operations):
def _package_update_insert_base(connection: Connection, package: Package) -> None: def _package_update_insert_base(connection: Connection, package: Package) -> None:
""" """
insert base package into table insert base package into table
:param connection: database connection
:param package: package properties Args:
connection(Connection): database connection
package(Package): package properties
""" """
connection.execute( connection.execute(
""" """
@ -81,8 +87,10 @@ class PackageOperations(Operations):
def _package_update_insert_packages(connection: Connection, package: Package) -> None: def _package_update_insert_packages(connection: Connection, package: Package) -> None:
""" """
insert packages into table insert packages into table
:param connection: database connection
:param package: package properties Args:
connection(Connection): database connection
package(Package): package properties
""" """
package_list = [] package_list = []
for name, description in package.packages.items(): for name, description in package.packages.items():
@ -108,9 +116,11 @@ class PackageOperations(Operations):
def _package_update_insert_status(connection: Connection, package_base: str, status: BuildStatus) -> None: def _package_update_insert_status(connection: Connection, package_base: str, status: BuildStatus) -> None:
""" """
insert base package status into table insert base package status into table
:param connection: database connection
:param package_base: package base name Args:
:param status: new build status connection(Connection): database connection
package_base(str): package base name
status(BuildStatus): new build status
""" """
connection.execute( connection.execute(
""" """
@ -126,8 +136,12 @@ class PackageOperations(Operations):
def _packages_get_select_package_bases(connection: Connection) -> Dict[str, Package]: def _packages_get_select_package_bases(connection: Connection) -> Dict[str, Package]:
""" """
select package bases from the table select package bases from the table
:param connection: database connection
:return: map of the package base to its descriptor (without packages themselves) Args:
connection(Connection): database connection
Returns:
Dict[str, Package]: map of the package base to its descriptor (without packages themselves)
""" """
return { return {
row["package_base"]: Package(row["package_base"], row["version"], row["aur_url"], {}) row["package_base"]: Package(row["package_base"], row["version"], row["aur_url"], {})
@ -138,9 +152,13 @@ class PackageOperations(Operations):
def _packages_get_select_packages(connection: Connection, packages: Dict[str, Package]) -> Dict[str, Package]: def _packages_get_select_packages(connection: Connection, packages: Dict[str, Package]) -> Dict[str, Package]:
""" """
select packages from the table select packages from the table
:param connection: database connection
:param packages: packages descriptor map Args:
:return: map of the package base to its descriptor including individual packages connection(Connection): database connection
packages(Dict[str, Package]): packages descriptor map
Returns:
Dict[str, Package]: map of the package base to its descriptor including individual packages
""" """
for row in connection.execute("""select * from packages"""): for row in connection.execute("""select * from packages"""):
if row["package_base"] not in packages: if row["package_base"] not in packages:
@ -152,8 +170,12 @@ class PackageOperations(Operations):
def _packages_get_select_statuses(connection: Connection) -> Dict[str, BuildStatus]: def _packages_get_select_statuses(connection: Connection) -> Dict[str, BuildStatus]:
""" """
select package build statuses from the table select package build statuses from the table
:param connection: database connection
:return: map of the package base to its status Args:
connection(Connection): database connection
Returns:
Dict[str, BuildStatus]: map of the package base to its status
""" """
return { return {
row["package_base"]: BuildStatus.from_json({"status": row["status"], "timestamp": row["last_updated"]}) row["package_base"]: BuildStatus.from_json({"status": row["status"], "timestamp": row["last_updated"]})
@ -163,7 +185,9 @@ class PackageOperations(Operations):
def package_remove(self, package_base: str) -> None: def package_remove(self, package_base: str) -> None:
""" """
remove package from database remove package from database
:param package_base: package base name
Args:
package_base(str): package base name
""" """
def run(connection: Connection) -> None: def run(connection: Connection) -> None:
self._package_remove_packages(connection, package_base, []) self._package_remove_packages(connection, package_base, [])
@ -174,8 +198,10 @@ class PackageOperations(Operations):
def package_update(self, package: Package, status: BuildStatus) -> None: def package_update(self, package: Package, status: BuildStatus) -> None:
""" """
update package status update package status
:param package: package properties
:param status: new build status Args:
package(Package): package properties
status(BuildStatus): new build status
""" """
def run(connection: Connection) -> None: def run(connection: Connection) -> None:
self._package_update_insert_base(connection, package) self._package_update_insert_base(connection, package)
@ -188,7 +214,9 @@ class PackageOperations(Operations):
def packages_get(self) -> List[Tuple[Package, BuildStatus]]: def packages_get(self) -> List[Tuple[Package, BuildStatus]]:
""" """
get package list and their build statuses from database get package list and their build statuses from database
:return: list of package properties and their statuses
Return:
List[Tuple[Package, BuildStatus]]: list of package properties and their statuses
""" """
def run(connection: Connection) -> Generator[Tuple[Package, BuildStatus], None, None]: def run(connection: Connection) -> Generator[Tuple[Package, BuildStatus], None, None]:
packages = self._packages_get_select_package_bases(connection) packages = self._packages_get_select_package_bases(connection)

View File

@ -31,16 +31,22 @@ class PatchOperations(Operations):
def patches_get(self, package_base: str) -> Optional[str]: def patches_get(self, package_base: str) -> Optional[str]:
""" """
retrieve patches for the package retrieve patches for the package
:param package_base: package base to search for patches
:return: plain text patch for the package Args:
package_base(str): package base to search for patches
Returns:
Optional[str]: plain text patch for the package
""" """
return self.patches_list(package_base).get(package_base) return self.patches_list(package_base).get(package_base)
def patches_insert(self, package_base: str, patch: str) -> None: def patches_insert(self, package_base: str, patch: str) -> None:
""" """
insert or update patch in database insert or update patch in database
:param package_base: package base to insert
:param patch: patch content Args:
package_base(str): package base to insert
patch(str): patch content
""" """
def run(connection: Connection) -> None: def run(connection: Connection) -> None:
connection.execute( connection.execute(
@ -59,8 +65,12 @@ class PatchOperations(Operations):
def patches_list(self, package_base: Optional[str]) -> Dict[str, str]: def patches_list(self, package_base: Optional[str]) -> Dict[str, str]:
""" """
extract all patches extract all patches
:param package_base: optional filter by package base
:return: map of package base to patch content Args:
package_base(Optional[str]): optional filter by package base
Returns:
Dict[str, str]: map of package base to patch content
""" """
def run(connection: Connection) -> Dict[str, str]: def run(connection: Connection) -> Dict[str, str]:
return { return {
@ -75,7 +85,9 @@ class PatchOperations(Operations):
def patches_remove(self, package_base: str) -> None: def patches_remove(self, package_base: str) -> None:
""" """
remove patch set remove patch set
:param package_base: package base to clear patches
Args:
package_base(str): package base to clear patches
""" """
def run(connection: Connection) -> None: def run(connection: Connection) -> None:
connection.execute( connection.execute(

View File

@ -22,6 +22,7 @@ from __future__ import annotations
import json import json
import sqlite3 import sqlite3
from pathlib import Path
from sqlite3 import Connection from sqlite3 import Connection
from typing import Type from typing import Type
@ -43,17 +44,37 @@ class SQLite(AuthOperations, BuildOperations, PackageOperations, PatchOperations
def load(cls: Type[SQLite], configuration: Configuration) -> SQLite: def load(cls: Type[SQLite], configuration: Configuration) -> SQLite:
""" """
construct instance from configuration construct instance from configuration
:param configuration: configuration instance
:return: fully initialized instance of the database Args:
configuration(Configuration): configuration instance
Returns:
SQLite: fully initialized instance of the database
""" """
database = cls(configuration.getpath("settings", "database")) path = cls.database_path(configuration)
database = cls(path)
database.init(configuration) database.init(configuration)
return database return database
@staticmethod
def database_path(configuration: Configuration) -> Path:
"""
read database from configuration
Args:
configuration(Configuration): configuration instance
Returns:
Path: database path according to the configuration
"""
return configuration.getpath("settings", "database")
def init(self, configuration: Configuration) -> None: def init(self, configuration: Configuration) -> None:
""" """
perform database migrations perform database migrations
:param configuration: configuration instance
Args:
configuration(Configuration): configuration instance
""" """
# custom types support # custom types support
sqlite3.register_adapter(dict, json.dumps) sqlite3.register_adapter(dict, json.dumps)

View File

@ -29,7 +29,9 @@ class BuildFailed(RuntimeError):
def __init__(self, package_base: str) -> None: def __init__(self, package_base: str) -> None:
""" """
default constructor default constructor
:param package_base: package base raised exception
Args:
package_base(str): package base raised exception
""" """
RuntimeError.__init__(self, f"Package {package_base} build failed, check logs for details") RuntimeError.__init__(self, f"Package {package_base} build failed, check logs for details")
@ -61,7 +63,9 @@ class InitializeException(RuntimeError):
def __init__(self, details: str) -> None: def __init__(self, details: str) -> None:
""" """
default constructor default constructor
:param details: details of the exception
Args:
details(str): details of the exception
""" """
RuntimeError.__init__(self, f"Could not load service: {details}") RuntimeError.__init__(self, f"Could not load service: {details}")
@ -74,7 +78,9 @@ class InvalidOption(ValueError):
def __init__(self, value: Any) -> None: def __init__(self, value: Any) -> None:
""" """
default constructor default constructor
:param value: option value
Args:
value(Any): option value
""" """
ValueError.__init__(self, f"Invalid or unknown option value `{value}`") ValueError.__init__(self, f"Invalid or unknown option value `{value}`")
@ -87,8 +93,10 @@ class InvalidPath(ValueError):
def __init__(self, path: Path, root: Path) -> None: def __init__(self, path: Path, root: Path) -> None:
""" """
default constructor default constructor
:param path: path which raised an exception
:param root: repository root (i.e. ahriman home) Args:
path(Path): path which raised an exception
root(Path): repository root (i.e. ahriman home)
""" """
ValueError.__init__(self, f"Path `{path}` does not belong to repository root `{root}`") ValueError.__init__(self, f"Path `{path}` does not belong to repository root `{root}`")
@ -101,7 +109,9 @@ class InvalidPackageInfo(RuntimeError):
def __init__(self, details: Any) -> None: def __init__(self, details: Any) -> None:
""" """
default constructor default constructor
:param details: error details
Args:
details(Any): error details
""" """
RuntimeError.__init__(self, f"There are errors during reading package information: `{details}`") RuntimeError.__init__(self, f"There are errors during reading package information: `{details}`")
@ -114,7 +124,9 @@ class MigrationError(RuntimeError):
def __init__(self, details: str) -> None: def __init__(self, details: str) -> None:
""" """
default constructor default constructor
:param details: error details
Args:
details(str): error details
""" """
RuntimeError.__init__(self, details) RuntimeError.__init__(self, details)
@ -127,7 +139,9 @@ class MissingArchitecture(ValueError):
def __init__(self, command: str) -> None: def __init__(self, command: str) -> None:
""" """
default constructor default constructor
:param command: command name which throws exception
Args:
command(str): command name which throws exception
""" """
ValueError.__init__(self, f"Architecture required for subcommand {command}, but missing") ValueError.__init__(self, f"Architecture required for subcommand {command}, but missing")
@ -140,7 +154,9 @@ class MultipleArchitectures(ValueError):
def __init__(self, command: str) -> None: def __init__(self, command: str) -> None:
""" """
default constructor default constructor
:param command: command name which throws exception
Args:
command(str): command name which throws exception
""" """
ValueError.__init__(self, f"Multiple architectures are not supported by subcommand {command}") ValueError.__init__(self, f"Multiple architectures are not supported by subcommand {command}")
@ -165,7 +181,9 @@ class SuccessFailed(ValueError):
def __init__(self, package_base: str) -> None: def __init__(self, package_base: str) -> None:
""" """
default constructor default constructor
:param package_base: package base name
Args:
package_base(str): package base name
""" """
ValueError.__init__(self, f"Package base {package_base} had status failed, but new status is success") ValueError.__init__(self, f"Package base {package_base} had status failed, but new status is success")
@ -190,7 +208,9 @@ class UnknownPackage(ValueError):
def __init__(self, package_base: str) -> None: def __init__(self, package_base: str) -> None:
""" """
default constructor default constructor
:param package_base: package base name
Args:
package_base(str): package base name
""" """
ValueError.__init__(self, f"Package base {package_base} is unknown") ValueError.__init__(self, f"Package base {package_base} is unknown")
@ -203,8 +223,10 @@ class UnsafeRun(RuntimeError):
def __init__(self, current_uid: int, root_uid: int) -> None: def __init__(self, current_uid: int, root_uid: int) -> None:
""" """
default constructor default constructor
:param current_uid: current user ID
:param root_uid: ID of the owner of root directory Args:
current_uid(int): current user ID
root_uid(int): ID of the owner of root directory
""" """
RuntimeError.__init__(self, f"Current UID {current_uid} differs from root owner {root_uid}. " RuntimeError.__init__(self, f"Current UID {current_uid} differs from root owner {root_uid}. "
f"Note that for the most actions it is unsafe to run application as different user." f"Note that for the most actions it is unsafe to run application as different user."

View File

@ -28,13 +28,17 @@ from ahriman.models.property import Property
class AurPrinter(StringPrinter): class AurPrinter(StringPrinter):
""" """
print content of the AUR package print content of the AUR package
:ivar package: AUR package description
Attributes:
package(AURPackage): AUR package description
""" """
def __init__(self, package: AURPackage) -> None: def __init__(self, package: AURPackage) -> None:
""" """
default constructor default constructor
:param package: AUR package description
Args:
package(AURPackage): AUR package description
""" """
StringPrinter.__init__(self, f"{package.name} {package.version} ({package.num_votes})") StringPrinter.__init__(self, f"{package.name} {package.version} ({package.num_votes})")
self.package = package self.package = package
@ -42,7 +46,9 @@ class AurPrinter(StringPrinter):
def properties(self) -> List[Property]: def properties(self) -> List[Property]:
""" """
convert content into printable data convert content into printable data
:return: list of content properties
Returns:
List[Property]: list of content properties
""" """
return [ return [
Property("Package base", self.package.package_base), Property("Package base", self.package.package_base),

View File

@ -29,9 +29,11 @@ class BuildPrinter(StringPrinter):
def __init__(self, package: Package, is_success: bool, use_utf: bool) -> None: def __init__(self, package: Package, is_success: bool, use_utf: bool) -> None:
""" """
default constructor default constructor
:param package: built package
:param is_success: True in case if build has success status and False otherwise Args:
:param use_utf: use utf instead of normal symbols package(Package): built package
is_success(bool): True in case if build has success status and False otherwise
use_utf(bool): use utf instead of normal symbols
""" """
StringPrinter.__init__(self, f"{self.sign(is_success, use_utf)} {package.base}") StringPrinter.__init__(self, f"{self.sign(is_success, use_utf)} {package.base}")
@ -39,9 +41,13 @@ class BuildPrinter(StringPrinter):
def sign(is_success: bool, use_utf: bool) -> str: def sign(is_success: bool, use_utf: bool) -> str:
""" """
generate sign according to settings generate sign according to settings
:param use_utf: use utf instead of normal symbols
:param is_success: True in case if build has success status and False otherwise Args:
:return: sign symbol according to current settings is_success(bool): True in case if build has success status and False otherwise
use_utf(bool): use utf instead of normal symbols
Returns:
str: sign symbol according to current settings
""" """
if is_success: if is_success:
return "[✔]" if use_utf else "[x]" return "[✔]" if use_utf else "[x]"

View File

@ -26,14 +26,18 @@ from ahriman.models.property import Property
class ConfigurationPrinter(StringPrinter): class ConfigurationPrinter(StringPrinter):
""" """
print content of the configuration section print content of the configuration section
:ivar values: configuration values dictionary
Attributes:
values(Dict[str, str]): configuration values dictionary
""" """
def __init__(self, section: str, values: Dict[str, str]) -> None: def __init__(self, section: str, values: Dict[str, str]) -> None:
""" """
default constructor default constructor
:param section: section name
:param values: configuration values dictionary Args:
section(str): section name
values(Dict[str, str]): configuration values dictionary
""" """
StringPrinter.__init__(self, f"[{section}]") StringPrinter.__init__(self, f"[{section}]")
self.values = values self.values = values
@ -41,7 +45,9 @@ class ConfigurationPrinter(StringPrinter):
def properties(self) -> List[Property]: def properties(self) -> List[Property]:
""" """
convert content into printable data convert content into printable data
:return: list of content properties
Returns:
List[Property]: list of content properties
""" """
return [ return [
Property(key, value, is_required=True) Property(key, value, is_required=True)

View File

@ -28,15 +28,19 @@ from ahriman.models.property import Property
class PackagePrinter(StringPrinter): class PackagePrinter(StringPrinter):
""" """
print content of the internal package object print content of the internal package object
:ivar package: package description
:ivar status: build status Attributes:
package(Package): package description
status(BuildStatus): build status
""" """
def __init__(self, package: Package, status: BuildStatus) -> None: def __init__(self, package: Package, status: BuildStatus) -> None:
""" """
default constructor default constructor
:param package: package description
:param status: build status Args:
package(Package): package description
status(BuildStatus): build status
""" """
StringPrinter.__init__(self, package.pretty_print()) StringPrinter.__init__(self, package.pretty_print())
self.package = package self.package = package
@ -45,7 +49,9 @@ class PackagePrinter(StringPrinter):
def properties(self) -> List[Property]: def properties(self) -> List[Property]:
""" """
convert content into printable data convert content into printable data
:return: list of content properties
Returns:
List[Property]: list of content properties
""" """
return [ return [
Property("Version", self.package.version, is_required=True), Property("Version", self.package.version, is_required=True),

View File

@ -30,9 +30,12 @@ class Printer:
def print(self, verbose: bool, log_fn: Callable[[str], None] = print, separator: str = ": ") -> None: def print(self, verbose: bool, log_fn: Callable[[str], None] = print, separator: str = ": ") -> None:
""" """
print content print content
:param verbose: print all fields
:param log_fn: logger function to log data Args:
:param separator: separator for property name and property value verbose(bool): print all fields
log_fn(Callable[[str]): logger function to log data
None]: (Default value = print)
separator(str, optional): separator for property name and property value (Default value = ": ")
""" """
if (title := self.title()) is not None: if (title := self.title()) is not None:
log_fn(title) log_fn(title)
@ -44,12 +47,16 @@ class Printer:
def properties(self) -> List[Property]: # pylint: disable=no-self-use def properties(self) -> List[Property]: # pylint: disable=no-self-use
""" """
convert content into printable data convert content into printable data
:return: list of content properties
Returns:
List[Property]: list of content properties
""" """
return [] return []
def title(self) -> Optional[str]: def title(self) -> Optional[str]:
""" """
generate entry title from content generate entry title from content
:return: content title if it can be generated and None otherwise
Returns:
Optional[str]: content title if it can be generated and None otherwise
""" """

View File

@ -29,6 +29,8 @@ class StatusPrinter(StringPrinter):
def __init__(self, status: BuildStatus) -> None: def __init__(self, status: BuildStatus) -> None:
""" """
default constructor default constructor
:param status: build status
Args:
status(BuildStatus): build status
""" """
StringPrinter.__init__(self, status.pretty_print()) StringPrinter.__init__(self, status.pretty_print())

View File

@ -30,13 +30,17 @@ class StringPrinter(Printer):
def __init__(self, content: str) -> None: def __init__(self, content: str) -> None:
""" """
default constructor default constructor
:param content: any content string
Args:
content(str): any content string
""" """
self.content = content self.content = content
def title(self) -> Optional[str]: def title(self) -> Optional[str]:
""" """
generate entry title from content generate entry title from content
:return: content title if it can be generated and None otherwise
Returns:
Optional[str]: content title if it can be generated and None otherwise
""" """
return self.content return self.content

View File

@ -27,15 +27,19 @@ from ahriman.models.property import Property
class UpdatePrinter(StringPrinter): class UpdatePrinter(StringPrinter):
""" """
print content of the package update print content of the package update
:ivar package: remote (new) package object
:ivar local_version: local version of the package if any Attributes:
package(Package): remote (new) package object
local_version(Optional[str]): local version of the package if any
""" """
def __init__(self, remote: Package, local_version: Optional[str]) -> None: def __init__(self, remote: Package, local_version: Optional[str]) -> None:
""" """
default constructor default constructor
:param remote: remote (new) package object
:param local_version: local version of the package if any Args:
remote(Package): remote (new) package object
local_version(Optional[str]): local version of the package if any
""" """
StringPrinter.__init__(self, remote.base) StringPrinter.__init__(self, remote.base)
self.package = remote self.package = remote
@ -44,6 +48,8 @@ class UpdatePrinter(StringPrinter):
def properties(self) -> List[Property]: def properties(self) -> List[Property]:
""" """
convert content into printable data convert content into printable data
:return: list of content properties
Returns:
List[Property]: list of content properties
""" """
return [Property(self.local_version, self.package.version, is_required=True)] return [Property(self.local_version, self.package.version, is_required=True)]

View File

@ -27,13 +27,17 @@ from ahriman.models.user import User
class UserPrinter(StringPrinter): class UserPrinter(StringPrinter):
""" """
print properties of user print properties of user
:ivar user: stored user
Attributes:
user(User): stored user
""" """
def __init__(self, user: User) -> None: def __init__(self, user: User) -> None:
""" """
default constructor default constructor
:param user: user to print
Args:
user(User): user to print
""" """
StringPrinter.__init__(self, user.username) StringPrinter.__init__(self, user.username)
self.user = user self.user = user
@ -41,6 +45,8 @@ class UserPrinter(StringPrinter):
def properties(self) -> List[Property]: def properties(self) -> List[Property]:
""" """
convert content into printable data convert content into printable data
:return: list of content properties
Returns:
List[Property]: list of content properties
""" """
return [Property("role", self.user.access.value, is_required=True)] return [Property("role", self.user.access.value, is_required=True)]

View File

@ -29,15 +29,19 @@ from ahriman.models.result import Result
class Console(Report): class Console(Report):
""" """
html report generator html report generator
:ivar use_utf: print utf8 symbols instead of ASCII
Attributes:
use_utf(bool): print utf8 symbols instead of ASCII
""" """
def __init__(self, architecture: str, configuration: Configuration, section: str) -> None: def __init__(self, architecture: str, configuration: Configuration, section: str) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture
:param configuration: configuration instance Args:
:param section: settings section name architecture(str): repository architecture
configuration(Configuration): configuration instance
section(str): settings section name
""" """
Report.__init__(self, architecture, configuration) Report.__init__(self, architecture, configuration)
self.use_utf = configuration.getboolean(section, "use_utf", fallback=True) self.use_utf = configuration.getboolean(section, "use_utf", fallback=True)
@ -45,8 +49,10 @@ class Console(Report):
def generate(self, packages: Iterable[Package], result: Result) -> None: def generate(self, packages: Iterable[Package], result: Result) -> None:
""" """
generate report for the specified packages generate report for the specified packages
:param packages: list of packages to generate report
:param result: build result Args:
packages(Iterable[Package]): list of packages to generate report
result(Result): build result
""" """
for package in result.success: for package in result.success:
BuildPrinter(package, is_success=True, use_utf=self.use_utf).print(verbose=True) BuildPrinter(package, is_success=True, use_utf=self.use_utf).print(verbose=True)

View File

@ -36,24 +36,28 @@ from ahriman.models.smtp_ssl_settings import SmtpSSLSettings
class Email(Report, JinjaTemplate): class Email(Report, JinjaTemplate):
""" """
email report generator email report generator
:ivar full_template_path: path to template for full package list
:ivar host: SMTP host to connect Attributes:
:ivar no_empty_report: skip empty report generation full_template_path(Path): path to template for full package list
:ivar password: password to authenticate via SMTP host(str): SMTP host to connect
:ivar port: SMTP port to connect no_empty_report(bool): skip empty report generation
:ivar receivers: list of receivers emails password(Optional[str]): password to authenticate via SMTP
:ivar sender: sender email address port(int): SMTP port to connect
:ivar ssl: SSL mode for SMTP connection receivers(List[str]): list of receivers emails
:ivar template_path: path to template for built packages sender(str): sender email address
:ivar user: username to authenticate via SMTP ssl(SmtpSSLSettings): SSL mode for SMTP connection
template_path(Path): path to template for built packages
user(Optional[str]): username to authenticate via SMTP
""" """
def __init__(self, architecture: str, configuration: Configuration, section: str) -> None: def __init__(self, architecture: str, configuration: Configuration, section: str) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture
:param configuration: configuration instance Args:
:param section: settings section name architecture(str): repository architecture
configuration(Configuration): configuration instance
section(str): settings section name
""" """
Report.__init__(self, architecture, configuration) Report.__init__(self, architecture, configuration)
JinjaTemplate.__init__(self, section, configuration) JinjaTemplate.__init__(self, section, configuration)
@ -74,8 +78,10 @@ class Email(Report, JinjaTemplate):
def _send(self, text: str, attachment: Dict[str, str]) -> None: def _send(self, text: str, attachment: Dict[str, str]) -> None:
""" """
send email callback send email callback
:param text: email body text
:param attachment: map of attachment filename to attachment content Args:
text(str): email body text
attachment(Dict[str, str]): map of attachment filename to attachment content
""" """
message = MIMEMultipart() message = MIMEMultipart()
message["From"] = self.sender message["From"] = self.sender
@ -102,8 +108,10 @@ class Email(Report, JinjaTemplate):
def generate(self, packages: Iterable[Package], result: Result) -> None: def generate(self, packages: Iterable[Package], result: Result) -> None:
""" """
generate report for the specified packages generate report for the specified packages
:param packages: list of packages to generate report
:param result: build result Args:
packages(Iterable[Package]): list of packages to generate report
result(Result): build result
""" """
if self.no_empty_report and not result.success: if self.no_empty_report and not result.success:
return return

View File

@ -29,16 +29,20 @@ from ahriman.models.result import Result
class HTML(Report, JinjaTemplate): class HTML(Report, JinjaTemplate):
""" """
html report generator html report generator
:ivar report_path: output path to html report
:ivar template_path: path to template for full package list Attributes:
report_path(Path): output path to html report
template_path(Path): path to template for full package list
""" """
def __init__(self, architecture: str, configuration: Configuration, section: str) -> None: def __init__(self, architecture: str, configuration: Configuration, section: str) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture
:param configuration: configuration instance Args:
:param section: settings section name architecture(str): repository architecture
configuration(Configuration): configuration instance
section(str): settings section name
""" """
Report.__init__(self, architecture, configuration) Report.__init__(self, architecture, configuration)
JinjaTemplate.__init__(self, section, configuration) JinjaTemplate.__init__(self, section, configuration)
@ -49,8 +53,10 @@ class HTML(Report, JinjaTemplate):
def generate(self, packages: Iterable[Package], result: Result) -> None: def generate(self, packages: Iterable[Package], result: Result) -> None:
""" """
generate report for the specified packages generate report for the specified packages
:param packages: list of packages to generate report
:param result: build result Args:
packages(Iterable[Package]): list of packages to generate report
result(Result): build result
""" """
html = self.make_html(Result(success=packages), self.template_path) html = self.make_html(Result(success=packages), self.template_path)
self.report_path.write_text(html) self.report_path.write_text(html)

View File

@ -35,38 +35,41 @@ class JinjaTemplate:
It uses jinja2 templates for report generation, the following variables are allowed: It uses jinja2 templates for report generation, the following variables are allowed:
homepage - link to homepage, string, optional * homepage - link to homepage, string, optional
link_path - prefix fo packages to download, string, required * link_path - prefix fo packages to download, string, required
has_package_signed - True in case if package sign enabled, False otherwise, required * has_package_signed - True in case if package sign enabled, False otherwise, required
has_repo_signed - True in case if repository database sign enabled, False otherwise, required * has_repo_signed - True in case if repository database sign enabled, False otherwise, required
packages - sorted list of packages properties, required * packages - sorted list of packages properties, required
* architecture, string * architecture, string
* archive_size, pretty printed size, string * archive_size, pretty printed size, string
* build_date, pretty printed datetime, string * build_date, pretty printed datetime, string
* depends, sorted list of strings * depends, sorted list of strings
* description, string * description, string
* filename, string, * filename, string,
* groups, sorted list of strings * groups, sorted list of strings
* installed_size, pretty printed datetime, string * installed_size, pretty printed datetime, string
* licenses, sorted list of strings * licenses, sorted list of strings
* name, string * name, string
* url, string * url, string
* version, string * version, string
pgp_key - default PGP key ID, string, optional * pgp_key - default PGP key ID, string, optional
repository - repository name, string, required * repository - repository name, string, required
:ivar homepage: homepage link if any (for footer) Attributes:
:ivar link_path: prefix fo packages to download homepage(Optional[str]): homepage link if any (for footer)
:ivar name: repository name link_path(str): prefix fo packages to download
:ivar default_pgp_key: default PGP key name(str): repository name
:ivar sign_targets: targets to sign enabled in configuration default_pgp_key(Optional[str]): default PGP key
sign_targets(Set[SignSettings]): targets to sign enabled in configuration
""" """
def __init__(self, section: str, configuration: Configuration) -> None: def __init__(self, section: str, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param section: settings section name
:param configuration: configuration instance Args:
section(str): settings section name
configuration(Configuration): configuration instance
""" """
self.link_path = configuration.get(section, "link_path") self.link_path = configuration.get(section, "link_path")
@ -79,8 +82,10 @@ class JinjaTemplate:
def make_html(self, result: Result, template_path: Path) -> str: def make_html(self, result: Result, template_path: Path) -> str:
""" """
generate report for the specified packages generate report for the specified packages
:param result: build result
:param template_path: path to jinja template Args:
result(Result): build result
template_path(Path): path to jinja template
""" """
# idea comes from https://stackoverflow.com/a/38642558 # idea comes from https://stackoverflow.com/a/38642558
loader = jinja2.FileSystemLoader(searchpath=template_path.parent) loader = jinja2.FileSystemLoader(searchpath=template_path.parent)
@ -101,7 +106,7 @@ class JinjaTemplate:
"name": package, "name": package,
"url": properties.url or "", "url": properties.url or "",
"version": base.version "version": base.version
} for base in result.updated for package, properties in base.packages.items() } for base in result.success for package, properties in base.packages.items()
] ]
comparator: Callable[[Dict[str, str]], str] = lambda item: item["filename"] comparator: Callable[[Dict[str, str]], str] = lambda item: item["filename"]

View File

@ -33,16 +33,20 @@ from ahriman.models.result import Result
class Report: class Report:
""" """
base report generator base report generator
:ivar architecture: repository architecture
:ivar configuration: configuration instance Attributes:
:ivar logger: class logger architecture(str): repository architecture
configuration(Configuration): configuration instance
logger(logging.Logger): class logger
""" """
def __init__(self, architecture: str, configuration: Configuration) -> None: def __init__(self, architecture: str, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture
:param configuration: configuration instance Args:
architecture(str): repository architecture
configuration(Configuration): configuration instance
""" """
self.logger = logging.getLogger("root") self.logger = logging.getLogger("root")
self.architecture = architecture self.architecture = architecture
@ -52,10 +56,14 @@ class Report:
def load(cls: Type[Report], architecture: str, configuration: Configuration, target: str) -> Report: def load(cls: Type[Report], architecture: str, configuration: Configuration, target: str) -> Report:
""" """
load client from settings load client from settings
:param architecture: repository architecture
:param configuration: configuration instance Args:
:param target: target to generate report aka section name (e.g. html) architecture(str): repository architecture
:return: client according to current settings configuration(Configuration): configuration instance
target(str): target to generate report aka section name (e.g. html)
Returns:
Report: client according to current settings
""" """
section, provider_name = configuration.gettype(target, architecture) section, provider_name = configuration.gettype(target, architecture)
provider = ReportSettings.from_option(provider_name) provider = ReportSettings.from_option(provider_name)
@ -68,20 +76,30 @@ class Report:
if provider == ReportSettings.Console: if provider == ReportSettings.Console:
from ahriman.core.report.console import Console from ahriman.core.report.console import Console
return Console(architecture, configuration, section) return Console(architecture, configuration, section)
if provider == ReportSettings.Telegram:
from ahriman.core.report.telegram import Telegram
return Telegram(architecture, configuration, section)
return cls(architecture, configuration) # should never happen return cls(architecture, configuration) # should never happen
def generate(self, packages: Iterable[Package], result: Result) -> None: def generate(self, packages: Iterable[Package], result: Result) -> None:
""" """
generate report for the specified packages generate report for the specified packages
:param packages: list of packages to generate report
:param result: build result Args:
packages(Iterable[Package]): list of packages to generate report
result(Result): build result
""" """
def run(self, packages: Iterable[Package], result: Result) -> None: def run(self, packages: Iterable[Package], result: Result) -> None:
""" """
run report generation run report generation
:param packages: list of packages to generate report
:param result: build result Args:
packages(Iterable[Package]): list of packages to generate report
result(Result): build result
Raises:
ReportFailed: in case of any report unmatched exception
""" """
try: try:
self.generate(packages, result) self.generate(packages, result)

View File

@ -0,0 +1,103 @@
#
# Copyright (c) 2021-2022 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# technically we could use python-telegram-bot, but it is just a single request, cmon
import requests
from typing import Iterable
from ahriman.core.configuration import Configuration
from ahriman.core.report.jinja_template import JinjaTemplate
from ahriman.core.report.report import Report
from ahriman.core.util import exception_response_text
from ahriman.models.package import Package
from ahriman.models.result import Result
class Telegram(Report, JinjaTemplate):
"""
telegram report generator
Attributes:
TELEGRAM_API_URL(str): (class attribute) telegram api base url
TELEGRAM_MAX_CONTENT_LENGTH(int): (class attribute) max content length of the message
api_key(str): bot api key
chat_id(str): chat id to post message, either string with @ or integer
template_path(Path): path to template for built packages
template_type(str): template message type to be used in parse mode, one of MarkdownV2, HTML, Markdown
"""
TELEGRAM_API_URL = "https://api.telegram.org"
TELEGRAM_MAX_CONTENT_LENGTH = 4096
def __init__(self, architecture: str, configuration: Configuration, section: str) -> None:
"""
default constructor
Args:
architecture(str): repository architecture
configuration(Configuration): configuration instance
section(str): settings section name
"""
Report.__init__(self, architecture, configuration)
JinjaTemplate.__init__(self, section, configuration)
self.api_key = configuration.get(section, "api_key")
self.chat_id = configuration.get(section, "chat_id")
self.template_path = configuration.getpath(section, "template_path")
self.template_type = configuration.get(section, "template_type", fallback="HTML")
def _send(self, text: str) -> None:
"""
send message to telegram channel
Args:
text(str): message body text
"""
try:
response = requests.post(
f"{self.TELEGRAM_API_URL}/bot{self.api_key}/sendMessage",
data={"chat_id": self.chat_id, "text": text, "parse_mode": self.template_type})
response.raise_for_status()
except requests.HTTPError as e:
self.logger.exception("could not perform request: %s", exception_response_text(e))
raise
except Exception:
self.logger.exception("could not perform request")
raise
def generate(self, packages: Iterable[Package], result: Result) -> None:
"""
generate report for the specified packages
Args:
packages(Iterable[Package]): list of packages to generate report
result(Result): build result
"""
if not result.success:
return
text = self.make_html(result, self.template_path)
# telegram content is limited by 4096 symbols, so we are going to split the message by new lines
# to fit into this restriction
if len(text) > self.TELEGRAM_MAX_CONTENT_LENGTH:
position = text.rfind("\n", 0, self.TELEGRAM_MAX_CONTENT_LENGTH)
portion, text = text[:position], text[position + 1:] # +1 to exclude newline we split
self._send(portion)
# send remaining (or full in case if size is less than max length) text
self._send(text)

View File

@ -33,7 +33,12 @@ class Cleaner(Properties):
def packages_built(self) -> List[Path]: def packages_built(self) -> List[Path]:
""" """
get list of files in built packages directory get list of files in built packages directory
:return: list of filenames from the directory
Returns:
List[Path]: list of filenames from the directory
Raises:
NotImplementedError: not implemented method
""" """
raise NotImplementedError raise NotImplementedError

View File

@ -39,23 +39,39 @@ class Executor(Cleaner):
def load_archives(self, packages: Iterable[Path]) -> List[Package]: def load_archives(self, packages: Iterable[Path]) -> List[Package]:
""" """
load packages from list of archives load packages from list of archives
:param packages: paths to package archives
:return: list of read packages Args:
packages(Iterable[Path]): paths to package archives
Returns:
List[Package]: list of read packages
Raises:
NotImplementedError: not implemented method
""" """
raise NotImplementedError raise NotImplementedError
def packages(self) -> List[Package]: def packages(self) -> List[Package]:
""" """
generate list of repository packages generate list of repository packages
:return: list of packages properties
Returns:
List[Package]: list of packages properties
Raises:
NotImplementedError: not implemented method
""" """
raise NotImplementedError raise NotImplementedError
def process_build(self, updates: Iterable[Package]) -> Result: def process_build(self, updates: Iterable[Package]) -> Result:
""" """
build packages build packages
:param updates: list of packages properties to build
:return: `packages_built` Args:
updates(Iterable[Package]): list of packages properties to build
Returns:
Result: build result
""" """
def build_single(package: Package, local_path: Path) -> None: def build_single(package: Package, local_path: Path) -> None:
self.reporter.set_building(package.base) self.reporter.set_building(package.base)
@ -82,8 +98,12 @@ class Executor(Cleaner):
def process_remove(self, packages: Iterable[str]) -> Path: def process_remove(self, packages: Iterable[str]) -> Path:
""" """
remove packages from list remove packages from list
:param packages: list of package names or bases to remove
:return: path to repository database Args:
packages(Iterable[str]): list of package names or bases to remove
Returns:
Path: path to repository database
""" """
def remove_base(package_base: str) -> None: def remove_base(package_base: str) -> None:
try: try:
@ -126,8 +146,11 @@ class Executor(Cleaner):
def process_report(self, targets: Optional[Iterable[str]], result: Result) -> None: def process_report(self, targets: Optional[Iterable[str]], result: Result) -> None:
""" """
generate reports generate reports
:param targets: list of targets to generate reports. Configuration option will be used if it is not set
:param result: build result Args:
targets(Optional[Iterable[str]]): list of targets to generate reports. Configuration option will be used
if it is not set
result(Result): build result
""" """
if targets is None: if targets is None:
targets = self.configuration.getlist("report", "target") targets = self.configuration.getlist("report", "target")
@ -138,8 +161,11 @@ class Executor(Cleaner):
def process_sync(self, targets: Optional[Iterable[str]], built_packages: Iterable[Package]) -> None: def process_sync(self, targets: Optional[Iterable[str]], built_packages: Iterable[Package]) -> None:
""" """
process synchronization to remote servers process synchronization to remote servers
:param targets: list of targets to sync. Configuration option will be used if it is not set
:param built_packages: list of packages which has just been built Args:
targets(Optional[Iterable[str]]): list of targets to sync. Configuration option will be used
if it is not set
built_packages(Iterable[Package]): list of packages which has just been built
""" """
if targets is None: if targets is None:
targets = self.configuration.getlist("upload", "target") targets = self.configuration.getlist("upload", "target")
@ -150,8 +176,12 @@ class Executor(Cleaner):
def process_update(self, packages: Iterable[Path]) -> Result: def process_update(self, packages: Iterable[Path]) -> Result:
""" """
sign packages, add them to repository and update repository database sign packages, add them to repository and update repository database
:param packages: list of filenames to run
:return: path to repository database Args:
packages(Iterable[Path]): list of filenames to run
Returns:
Result: path to repository database
""" """
def update_single(name: Optional[str], base: str) -> None: def update_single(name: Optional[str], base: str) -> None:
if name is None: if name is None:

View File

@ -32,29 +32,33 @@ from ahriman.core.util import check_user
class Properties: class Properties:
""" """
repository internal objects holder repository internal objects holder
:ivar architecture: repository architecture
:ivar aur_url: base AUR url Attributes:
:ivar configuration: configuration instance architecture(str): repository architecture
:ivar database: database instance aur_url(str): base AUR url
:ivar ignore_list: package bases which will be ignored during auto updates configuration(Configuration): configuration instance
:ivar logger: class logger database(SQLite): database instance
:ivar name: repository name ignore_list(List[str]): package bases which will be ignored during auto updates
:ivar pacman: alpm wrapper instance logger(logging.Logger): class logger
:ivar paths: repository paths instance name(str): repository name
:ivar repo: repo commands wrapper instance pacman(Pacman): alpm wrapper instance
:ivar reporter: build status reporter instance paths(RepositoryPaths): repository paths instance
:ivar sign: GPG wrapper instance repo(Repo): repo commands wrapper instance
reporter(Client): build status reporter instance
sign(GPG): GPG wrapper instance
""" """
def __init__(self, architecture: str, configuration: Configuration, database: SQLite, def __init__(self, architecture: str, configuration: Configuration, database: SQLite,
no_report: bool, unsafe: bool) -> None: no_report: bool, unsafe: bool) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture
:param configuration: configuration instance Args:
:param database: database instance architecture(str): repository architecture
:param no_report: force disable reporting configuration(Configuration): configuration instance
:param unsafe: if set no user check will be performed before path creation database(SQLite): database instance
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
""" """
self.logger = logging.getLogger("root") self.logger = logging.getLogger("root")
self.architecture = architecture self.architecture = architecture

View File

@ -35,8 +35,12 @@ class Repository(Executor, UpdateHandler):
def load_archives(self, packages: Iterable[Path]) -> List[Package]: def load_archives(self, packages: Iterable[Path]) -> List[Package]:
""" """
load packages from list of archives load packages from list of archives
:param packages: paths to package archives
:return: list of read packages Args:
packages(Iterable[Path]): paths to package archives
Returns:
List[Package]: list of read packages
""" """
result: Dict[str, Package] = {} result: Dict[str, Package] = {}
# we are iterating over bases, not single packages # we are iterating over bases, not single packages
@ -58,22 +62,30 @@ class Repository(Executor, UpdateHandler):
def packages(self) -> List[Package]: def packages(self) -> List[Package]:
""" """
generate list of repository packages generate list of repository packages
:return: list of packages properties
Returns:
List[Package]: list of packages properties
""" """
return self.load_archives(filter(package_like, self.paths.repository.iterdir())) return self.load_archives(filter(package_like, self.paths.repository.iterdir()))
def packages_built(self) -> List[Path]: def packages_built(self) -> List[Path]:
""" """
get list of files in built packages directory get list of files in built packages directory
:return: list of filenames from the directory
Returns:
List[Path]: list of filenames from the directory
""" """
return list(filter(package_like, self.paths.packages.iterdir())) return list(filter(package_like, self.paths.packages.iterdir()))
def packages_depends_on(self, depends_on: Optional[Iterable[str]]) -> List[Package]: def packages_depends_on(self, depends_on: Optional[Iterable[str]]) -> List[Package]:
""" """
extract list of packages which depends on specified package extract list of packages which depends on specified package
:param: depends_on: dependencies of the packages
:return: list of repository packages which depend on specified packages Args:
depends_on(Optional[Iterable[str]]): dependencies of the packages
Returns:
List[Package]: list of repository packages which depend on specified packages
""" """
packages = self.packages() packages = self.packages()
if depends_on is None: if depends_on is None:

View File

@ -33,16 +33,25 @@ class UpdateHandler(Cleaner):
def packages(self) -> List[Package]: def packages(self) -> List[Package]:
""" """
generate list of repository packages generate list of repository packages
:return: list of packages properties
Returns:
List[Package]: list of packages properties
Raises:
NotImplementedError: not implemented method
""" """
raise NotImplementedError raise NotImplementedError
def updates_aur(self, filter_packages: Iterable[str], no_vcs: bool) -> List[Package]: def updates_aur(self, filter_packages: Iterable[str], no_vcs: bool) -> List[Package]:
""" """
check AUR for updates check AUR for updates
:param filter_packages: do not check every package just specified in the list
:param no_vcs: do not check VCS packages Args:
:return: list of packages which are out-of-dated filter_packages(Iterable[str]): do not check every package just specified in the list
no_vcs(bool): do not check VCS packages
Returns:
List[Package]: list of packages which are out-of-dated
""" """
result: List[Package] = [] result: List[Package] = []
@ -70,7 +79,9 @@ class UpdateHandler(Cleaner):
def updates_local(self) -> List[Package]: def updates_local(self) -> List[Package]:
""" """
check local packages for updates check local packages for updates
:return: list of local packages which are out-of-dated
Returns:
List[Package]: list of local packages which are out-of-dated
""" """
result: List[Package] = [] result: List[Package] = []
packages = {local.base: local for local in self.packages()} packages = {local.base: local for local in self.packages()}
@ -97,7 +108,9 @@ class UpdateHandler(Cleaner):
def updates_manual(self) -> List[Package]: def updates_manual(self) -> List[Package]:
""" """
check for packages for which manual update has been requested check for packages for which manual update has been requested
:return: list of packages which are out-of-dated
Returns:
List[Package]: list of packages which are out-of-dated
""" """
result: List[Package] = [] result: List[Package] = []
known_bases = {package.base for package in self.packages()} known_bases = {package.base for package in self.packages()}

View File

@ -32,11 +32,13 @@ from ahriman.models.sign_settings import SignSettings
class GPG: class GPG:
""" """
gnupg wrapper gnupg wrapper
:ivar architecture: repository architecture
:ivar configuration: configuration instance Attributes:
:ivar default_key: default PGP key ID to use architecture(str): repository architecture
:ivar logger: class logger configuration(Configuration): configuration instance
:ivar targets: list of targets to sign (repository, package etc) default_key(Optional[str]): default PGP key ID to use
logger(logging.Logger): class logger
targets(Set[SignSettings]): list of targets to sign (repository, package etc)
""" """
_check_output = check_output _check_output = check_output
@ -44,8 +46,10 @@ class GPG:
def __init__(self, architecture: str, configuration: Configuration) -> None: def __init__(self, architecture: str, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture
:param configuration: configuration instance Args:
architecture(str): repository architecture
configuration(Configuration): configuration instance
""" """
self.logger = logging.getLogger("build_details") self.logger = logging.getLogger("build_details")
self.architecture = architecture self.architecture = architecture
@ -55,7 +59,8 @@ class GPG:
@property @property
def repository_sign_args(self) -> List[str]: def repository_sign_args(self) -> List[str]:
""" """
:return: command line arguments for repo-add command to sign database Returns:
List[str]: command line arguments for repo-add command to sign database
""" """
if SignSettings.Repository not in self.targets: if SignSettings.Repository not in self.targets:
return [] return []
@ -68,9 +73,13 @@ class GPG:
def sign_command(path: Path, key: str) -> List[str]: def sign_command(path: Path, key: str) -> List[str]:
""" """
gpg command to run gpg command to run
:param path: path to file to sign
:param key: PGP key ID Args:
:return: gpg command with all required arguments path(Path): path to file to sign
key(str): PGP key ID
Returns:
List[str]: gpg command with all required arguments
""" """
return ["gpg", "-u", key, "-b", str(path)] return ["gpg", "-u", key, "-b", str(path)]
@ -78,22 +87,32 @@ class GPG:
def sign_options(configuration: Configuration) -> Tuple[Set[SignSettings], Optional[str]]: def sign_options(configuration: Configuration) -> Tuple[Set[SignSettings], Optional[str]]:
""" """
extract default sign options from configuration extract default sign options from configuration
:param configuration: configuration instance
:return: tuple of sign targets and default PGP key Args:
configuration(Configuration): configuration instance
Returns:
Tuple[Set[SignSettings], Optional[str]]: tuple of sign targets and default PGP key
""" """
targets = { targets: Set[SignSettings] = set()
SignSettings.from_option(option) for option in configuration.getlist("sign", "target"):
for option in configuration.getlist("sign", "target") target = SignSettings.from_option(option)
} if target == SignSettings.Disabled:
continue
targets.add(target)
default_key = configuration.get("sign", "key") if targets else None default_key = configuration.get("sign", "key") if targets else None
return targets, default_key return targets, default_key
def key_download(self, server: str, key: str) -> str: def key_download(self, server: str, key: str) -> str:
""" """
download key from public PGP server download key from public PGP server
:param server: public PGP server which will be used to download the key
:param key: key ID to download Args:
:return: key as plain text server(str): public PGP server which will be used to download the key
key(str): key ID to download
Returns:
str: key as plain text
""" """
key = key if key.startswith("0x") else f"0x{key}" key = key if key.startswith("0x") else f"0x{key}"
try: try:
@ -111,8 +130,10 @@ class GPG:
def key_import(self, server: str, key: str) -> None: def key_import(self, server: str, key: str) -> None:
""" """
import key to current user and sign it locally import key to current user and sign it locally
:param server: public PGP server which will be used to download the key
:param key: key ID to import Args:
server(str): public PGP server which will be used to download the key
key(str): key ID to import
""" """
key_body = self.key_download(server, key) key_body = self.key_download(server, key)
GPG._check_output("gpg", "--import", input_data=key_body, exception=None, logger=self.logger) GPG._check_output("gpg", "--import", input_data=key_body, exception=None, logger=self.logger)
@ -120,9 +141,13 @@ class GPG:
def process(self, path: Path, key: str) -> List[Path]: def process(self, path: Path, key: str) -> List[Path]:
""" """
gpg command wrapper gpg command wrapper
:param path: path to file to sign
:param key: PGP key ID Args:
:return: list of generated files including original file path(Path): path to file to sign
key(str): PGP key ID
Returns:
List[Path]: list of generated files including original file
""" """
GPG._check_output( GPG._check_output(
*GPG.sign_command(path, key), *GPG.sign_command(path, key),
@ -133,9 +158,13 @@ class GPG:
def process_sign_package(self, path: Path, base: str) -> List[Path]: def process_sign_package(self, path: Path, base: str) -> List[Path]:
""" """
sign package if required by configuration sign package if required by configuration
:param path: path to file to sign
:param base: package base required to check for key overrides Args:
:return: list of generated files including original file path(Path): path to file to sign
base(str): package base required to check for key overrides
Returns:
List[Path]: list of generated files including original file
""" """
if SignSettings.Packages not in self.targets: if SignSettings.Packages not in self.targets:
return [path] return [path]
@ -149,8 +178,12 @@ class GPG:
""" """
sign repository if required by configuration sign repository if required by configuration
:note: more likely you just want to pass `repository_sign_args` to repo wrapper :note: more likely you just want to pass `repository_sign_args` to repo wrapper
:param path: path to repository database
:return: list of generated files including original file Args:
path(Path): path to repository database
Returns:
List[Path]: list of generated files including original file
""" """
if SignSettings.Repository not in self.targets: if SignSettings.Repository not in self.targets:
return [path] return [path]

View File

@ -35,19 +35,23 @@ class Spawn(Thread):
""" """
helper to spawn external ahriman process helper to spawn external ahriman process
MUST NOT be used directly, the only one usage allowed is to spawn process from web services MUST NOT be used directly, the only one usage allowed is to spawn process from web services
:ivar active: map of active child processes required to avoid zombies
:ivar architecture: repository architecture Attributes:
:ivar configuration: configuration instance active(Dict[str, Process]): map of active child processes required to avoid zombies
:ivar logger: spawner logger architecture(str): repository architecture
:ivar queue: multiprocessing queue to read updates from processes configuration(Configuration): configuration instance
logger(logging.Logger): spawner logger
queue(Queue[Tuple[str, bool]]): multiprocessing queue to read updates from processes
""" """
def __init__(self, args_parser: argparse.ArgumentParser, architecture: str, configuration: Configuration) -> None: def __init__(self, args_parser: argparse.ArgumentParser, architecture: str, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param args_parser: command line parser for the application
:param architecture: repository architecture Args:
:param configuration: configuration instance args_parser(argparse.ArgumentParser): command line parser for the application
architecture(str): repository architecture
configuration(Configuration): configuration instance
""" """
Thread.__init__(self, name="spawn") Thread.__init__(self, name="spawn")
self.architecture = architecture self.architecture = architecture
@ -65,11 +69,13 @@ class Spawn(Thread):
process_id: str, queue: Queue[Tuple[str, bool]]) -> None: # pylint: disable=unsubscriptable-object process_id: str, queue: Queue[Tuple[str, bool]]) -> None: # pylint: disable=unsubscriptable-object
""" """
helper to run external process helper to run external process
:param callback: application run function (i.e. Handler.run method)
:param args: command line arguments Args:
:param architecture: repository architecture callback(Callable[[argparse.Namespace, str], bool]): application run function (i.e. Handler.run method)
:param process_id: process unique identifier args(argparse.Namespace): command line arguments
:param queue: output queue architecture(str): repository architecture
process_id(str): process unique identifier
queue(Queue[Tuple[str, bool]]): output queue
""" """
result = callback(args, architecture) result = callback(args, architecture)
queue.put((process_id, result)) queue.put((process_id, result))
@ -77,8 +83,10 @@ class Spawn(Thread):
def packages_add(self, packages: Iterable[str], now: bool) -> None: def packages_add(self, packages: Iterable[str], now: bool) -> None:
""" """
add packages add packages
:param packages: packages list to add
:param now: build packages now Args:
packages(Iterable[str]): packages list to add
now(bool): build packages now
""" """
kwargs = {"source": PackageSource.AUR.value} # avoid abusing by building non-aur packages kwargs = {"source": PackageSource.AUR.value} # avoid abusing by building non-aur packages
if now: if now:
@ -88,16 +96,20 @@ class Spawn(Thread):
def packages_remove(self, packages: Iterable[str]) -> None: def packages_remove(self, packages: Iterable[str]) -> None:
""" """
remove packages remove packages
:param packages: packages list to remove
Args:
packages(Iterable[str]): packages list to remove
""" """
self.spawn_process("remove", *packages) self.spawn_process("remove", *packages)
def spawn_process(self, command: str, *args: str, **kwargs: str) -> None: def spawn_process(self, command: str, *args: str, **kwargs: str) -> None:
""" """
spawn external ahriman process with supplied arguments spawn external ahriman process with supplied arguments
:param command: subcommand to run
:param args: positional command arguments Args:
:param kwargs: named command arguments command(str): subcommand to run
*args(str): positional command arguments
**kwargs(str): named command arguments
""" """
# default arguments # default arguments
arguments = ["--architecture", self.architecture] arguments = ["--architecture", self.architecture]

View File

@ -36,8 +36,12 @@ class Client:
def load(cls: Type[Client], configuration: Configuration) -> Client: def load(cls: Type[Client], configuration: Configuration) -> Client:
""" """
load client from settings load client from settings
:param configuration: configuration instance
:return: client according to current settings Args:
configuration(Configuration): configuration instance
Returns:
Client: client according to current settings
""" """
address = configuration.get("web", "address", fallback=None) address = configuration.get("web", "address", fallback=None)
host = configuration.get("web", "host", fallback=None) host = configuration.get("web", "host", fallback=None)
@ -50,15 +54,21 @@ class Client:
def add(self, package: Package, status: BuildStatusEnum) -> None: def add(self, package: Package, status: BuildStatusEnum) -> None:
""" """
add new package with status add new package with status
:param package: package properties
:param status: current package build status Args:
package(Package): package properties
status(BuildStatusEnum): current package build status
""" """
def get(self, base: Optional[str]) -> List[Tuple[Package, BuildStatus]]: # pylint: disable=no-self-use def get(self, base: Optional[str]) -> List[Tuple[Package, BuildStatus]]: # pylint: disable=no-self-use
""" """
get package status get package status
:param base: package base to get
:return: list of current package description and status if it has been found Args:
base(Optional[str]): package base to get
Returns:
List[Tuple[Package, BuildStatus]]: list of current package description and status if it has been found
""" """
del base del base
return [] return []
@ -66,67 +76,87 @@ class Client:
def get_internal(self) -> InternalStatus: # pylint: disable=no-self-use def get_internal(self) -> InternalStatus: # pylint: disable=no-self-use
""" """
get internal service status get internal service status
:return: current internal (web) service status
Returns:
InternalStatus: current internal (web) service status
""" """
return InternalStatus() return InternalStatus()
def get_self(self) -> BuildStatus: # pylint: disable=no-self-use def get_self(self) -> BuildStatus: # pylint: disable=no-self-use
""" """
get ahriman status itself get ahriman status itself
:return: current ahriman status
Returns:
BuildStatus: current ahriman status
""" """
return BuildStatus() return BuildStatus()
def remove(self, base: str) -> None: def remove(self, base: str) -> None:
""" """
remove packages from watcher remove packages from watcher
:param base: package base to remove
Args:
base(str): package base to remove
""" """
def update(self, base: str, status: BuildStatusEnum) -> None: def update(self, base: str, status: BuildStatusEnum) -> None:
""" """
update package build status. Unlike `add` it does not update package properties update package build status. Unlike `add` it does not update package properties
:param base: package base to update
:param status: current package build status Args:
base(str): package base to update
status(BuildStatusEnum): current package build status
""" """
def update_self(self, status: BuildStatusEnum) -> None: def update_self(self, status: BuildStatusEnum) -> None:
""" """
update ahriman status itself update ahriman status itself
:param status: current ahriman status
Args:
status(BuildStatusEnum): current ahriman status
""" """
def set_building(self, base: str) -> None: def set_building(self, base: str) -> None:
""" """
set package status to building set package status to building
:param base: package base to update
Args:
base(str): package base to update
""" """
return self.update(base, BuildStatusEnum.Building) return self.update(base, BuildStatusEnum.Building)
def set_failed(self, base: str) -> None: def set_failed(self, base: str) -> None:
""" """
set package status to failed set package status to failed
:param base: package base to update
Args:
base(str): package base to update
""" """
return self.update(base, BuildStatusEnum.Failed) return self.update(base, BuildStatusEnum.Failed)
def set_pending(self, base: str) -> None: def set_pending(self, base: str) -> None:
""" """
set package status to pending set package status to pending
:param base: package base to update
Args:
base(str): package base to update
""" """
return self.update(base, BuildStatusEnum.Pending) return self.update(base, BuildStatusEnum.Pending)
def set_success(self, package: Package) -> None: def set_success(self, package: Package) -> None:
""" """
set package status to success set package status to success
:param package: current package properties
Args:
package(Package): current package properties
""" """
return self.add(package, BuildStatusEnum.Success) return self.add(package, BuildStatusEnum.Success)
def set_unknown(self, package: Package) -> None: def set_unknown(self, package: Package) -> None:
""" """
set package status to unknown set package status to unknown
:param package: current package properties
Args:
package(Package): current package properties
""" """
return self.add(package, BuildStatusEnum.Unknown) return self.add(package, BuildStatusEnum.Unknown)

View File

@ -32,20 +32,24 @@ from ahriman.models.package import Package
class Watcher: class Watcher:
""" """
package status watcher package status watcher
:ivar architecture: repository architecture
:ivar database: database instance Attributes:
:ivar known: list of known packages. For the most cases `packages` should be used instead architecture(str): repository architecture
:ivar logger: class logger database(SQLite): database instance
:ivar repository: repository object known(Dict[str, Tuple[Package, BuildStatus]]): list of known packages. For the most cases `packages` should be used instead
:ivar status: daemon status logger(logging.Logger): class logger
repository(Repository): repository object
status(BuildStatus): daemon status
""" """
def __init__(self, architecture: str, configuration: Configuration, database: SQLite) -> None: def __init__(self, architecture: str, configuration: Configuration, database: SQLite) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture
:param configuration: configuration instance Args:
:param database: database instance architecture(str): repository architecture
configuration(Configuration): configuration instance
database(SQLite): database instance
""" """
self.logger = logging.getLogger("http") self.logger = logging.getLogger("http")
@ -59,14 +63,23 @@ class Watcher:
@property @property
def packages(self) -> List[Tuple[Package, BuildStatus]]: def packages(self) -> List[Tuple[Package, BuildStatus]]:
""" """
:return: list of packages together with their statuses Returns:
List[Tuple[Package, BuildStatus]]: list of packages together with their statuses
""" """
return list(self.known.values()) return list(self.known.values())
def get(self, base: str) -> Tuple[Package, BuildStatus]: def get(self, base: str) -> Tuple[Package, BuildStatus]:
""" """
get current package base build status get current package base build status
:return: package and its status
Args:
base(str): package base
Returns:
Tuple[Package, BuildStatus]: package and its status
Raises:
UnknownPackage: if no package found
""" """
try: try:
return self.known[base] return self.known[base]
@ -92,7 +105,9 @@ class Watcher:
def remove(self, package_base: str) -> None: def remove(self, package_base: str) -> None:
""" """
remove package base from known list if any remove package base from known list if any
:param package_base: package base
Args:
package_base(str): package base
""" """
self.known.pop(package_base, None) self.known.pop(package_base, None)
self.database.package_remove(package_base) self.database.package_remove(package_base)
@ -100,9 +115,14 @@ class Watcher:
def update(self, package_base: str, status: BuildStatusEnum, package: Optional[Package]) -> None: def update(self, package_base: str, status: BuildStatusEnum, package: Optional[Package]) -> None:
""" """
update package status and description update package status and description
:param package_base: package base to update
:param status: new build status Args:
:param package: optional new package description. In case if not set current properties will be used package_base(str): package base to update
status(BuildStatusEnum): new build status
package(Optional[Package]): optional new package description. In case if not set current properties will be used
Raises:
UnknownPackage: if no package found
""" """
if package is None: if package is None:
try: try:
@ -116,6 +136,8 @@ class Watcher:
def update_self(self, status: BuildStatusEnum) -> None: def update_self(self, status: BuildStatusEnum) -> None:
""" """
update service status update service status
:param status: new service status
Args:
status(BuildStatusEnum): new service status
""" """
self.status = BuildStatus(status) self.status = BuildStatus(status)

View File

@ -34,15 +34,19 @@ from ahriman.models.user import User
class WebClient(Client): class WebClient(Client):
""" """
build status reporter web client build status reporter web client
:ivar address: address of the web service
:ivar logger: class logger Attributes:
:ivar user: web service user descriptor address(str): address of the web service
logger(logging.Logger): class logger
user(Optional[User]): web service user descriptor
""" """
def __init__(self, configuration: Configuration) -> None: def __init__(self, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param configuration: configuration instance
Args:
configuration(Configuration): configuration instance
""" """
self.logger = logging.getLogger("http") self.logger = logging.getLogger("http")
self.address = self.parse_address(configuration) self.address = self.parse_address(configuration)
@ -56,21 +60,24 @@ class WebClient(Client):
@property @property
def _ahriman_url(self) -> str: def _ahriman_url(self) -> str:
""" """
:return: full url for web service for ahriman service itself Returns:
str: full url for web service for ahriman service itself
""" """
return f"{self.address}/status-api/v1/ahriman" return f"{self.address}/status-api/v1/ahriman"
@property @property
def _login_url(self) -> str: def _login_url(self) -> str:
""" """
:return: full url for web service to login Returns:
str: full url for web service to login
""" """
return f"{self.address}/user-api/v1/login" return f"{self.address}/user-api/v1/login"
@property @property
def _status_url(self) -> str: def _status_url(self) -> str:
""" """
:return: full url for web service for status Returns:
str: full url for web service for status
""" """
return f"{self.address}/status-api/v1/status" return f"{self.address}/status-api/v1/status"
@ -78,8 +85,12 @@ class WebClient(Client):
def parse_address(configuration: Configuration) -> str: def parse_address(configuration: Configuration) -> str:
""" """
parse address from configuration parse address from configuration
:param configuration: configuration instance
:return: valid http address Args:
configuration(Configuration): configuration instance
Returns:
str: valid http address
""" """
address = configuration.get("web", "address", fallback=None) address = configuration.get("web", "address", fallback=None)
if not address: if not address:
@ -112,16 +123,22 @@ class WebClient(Client):
def _package_url(self, base: str = "") -> str: def _package_url(self, base: str = "") -> str:
""" """
url generator url generator
:param base: package base to generate url
:return: full url of web service for specific package base Args:
base(str, optional): package base to generate url (Default value = "")
Returns:
str: full url of web service for specific package base
""" """
return f"{self.address}/status-api/v1/packages/{base}" return f"{self.address}/status-api/v1/packages/{base}"
def add(self, package: Package, status: BuildStatusEnum) -> None: def add(self, package: Package, status: BuildStatusEnum) -> None:
""" """
add new package with status add new package with status
:param package: package properties
:param status: current package build status Args:
package(Package): package properties
status(BuildStatusEnum): current package build status
""" """
payload = { payload = {
"status": status.value, "status": status.value,
@ -139,8 +156,12 @@ class WebClient(Client):
def get(self, base: Optional[str]) -> List[Tuple[Package, BuildStatus]]: def get(self, base: Optional[str]) -> List[Tuple[Package, BuildStatus]]:
""" """
get package status get package status
:param base: package base to get
:return: list of current package description and status if it has been found Args:
base(Optional[str]): package base to get
Returns:
List[Tuple[Package, BuildStatus]]: list of current package description and status if it has been found
""" """
try: try:
response = self.__session.get(self._package_url(base or "")) response = self.__session.get(self._package_url(base or ""))
@ -160,7 +181,9 @@ class WebClient(Client):
def get_internal(self) -> InternalStatus: def get_internal(self) -> InternalStatus:
""" """
get internal service status get internal service status
:return: current internal (web) service status
Returns:
InternalStatus: current internal (web) service status
""" """
try: try:
response = self.__session.get(self._status_url) response = self.__session.get(self._status_url)
@ -177,7 +200,9 @@ class WebClient(Client):
def get_self(self) -> BuildStatus: def get_self(self) -> BuildStatus:
""" """
get ahriman status itself get ahriman status itself
:return: current ahriman status
Returns:
BuildStatus: current ahriman status
""" """
try: try:
response = self.__session.get(self._ahriman_url) response = self.__session.get(self._ahriman_url)
@ -194,7 +219,9 @@ class WebClient(Client):
def remove(self, base: str) -> None: def remove(self, base: str) -> None:
""" """
remove packages from watcher remove packages from watcher
:param base: basename to remove
Args:
base(str): basename to remove
""" """
try: try:
response = self.__session.delete(self._package_url(base)) response = self.__session.delete(self._package_url(base))
@ -207,8 +234,10 @@ class WebClient(Client):
def update(self, base: str, status: BuildStatusEnum) -> None: def update(self, base: str, status: BuildStatusEnum) -> None:
""" """
update package build status. Unlike `add` it does not update package properties update package build status. Unlike `add` it does not update package properties
:param base: package base to update
:param status: current package build status Args:
base(str): package base to update
status(BuildStatusEnum): current package build status
""" """
payload = {"status": status.value} payload = {"status": status.value}
@ -223,7 +252,9 @@ class WebClient(Client):
def update_self(self, status: BuildStatusEnum) -> None: def update_self(self, status: BuildStatusEnum) -> None:
""" """
update ahriman status itself update ahriman status itself
:param status: current ahriman status
Args:
status(BuildStatusEnum): current ahriman status
""" """
payload = {"status": status.value} payload = {"status": status.value}

View File

@ -19,29 +19,30 @@
# #
from __future__ import annotations from __future__ import annotations
import shutil
import tempfile
from pathlib import Path
from typing import Iterable, List, Set, Type from typing import Iterable, List, Set, Type
from ahriman.core.build_tools.sources import Sources from ahriman.core.build_tools.sources import Sources
from ahriman.core.database.sqlite import SQLite from ahriman.core.database.sqlite import SQLite
from ahriman.core.util import tmpdir
from ahriman.models.package import Package from ahriman.models.package import Package
class Leaf: class Leaf:
""" """
tree leaf implementation tree leaf implementation
:ivar dependencies: list of package dependencies
:ivar package: leaf package properties Attributes:
dependencies(Set[str]): list of package dependencies
package(Package): leaf package properties
""" """
def __init__(self, package: Package, dependencies: Set[str]) -> None: def __init__(self, package: Package, dependencies: Set[str]) -> None:
""" """
default constructor default constructor
:param package: package properties
:param dependencies: package dependencies Args:
package(Package): package properties
dependencies(Set[str]): package dependencies
""" """
self.package = package self.package = package
self.dependencies = dependencies self.dependencies = dependencies
@ -49,7 +50,8 @@ class Leaf:
@property @property
def items(self) -> Iterable[str]: def items(self) -> Iterable[str]:
""" """
:return: packages containing in this leaf Returns:
Iterable[str]: packages containing in this leaf
""" """
return self.package.packages.keys() return self.package.packages.keys()
@ -57,23 +59,28 @@ class Leaf:
def load(cls: Type[Leaf], package: Package, database: SQLite) -> Leaf: def load(cls: Type[Leaf], package: Package, database: SQLite) -> Leaf:
""" """
load leaf from package with dependencies load leaf from package with dependencies
:param package: package properties
:param database: database instance Args:
:return: loaded class package(Package): package properties
database(SQLite): database instance
Returns:
Leaf: loaded class
""" """
clone_dir = Path(tempfile.mkdtemp()) with tmpdir() as clone_dir:
try:
Sources.load(clone_dir, package.git_url, database.patches_get(package.base)) Sources.load(clone_dir, package.git_url, database.patches_get(package.base))
dependencies = Package.dependencies(clone_dir) dependencies = Package.dependencies(clone_dir)
finally:
shutil.rmtree(clone_dir, ignore_errors=True)
return cls(package, dependencies) return cls(package, dependencies)
def is_root(self, packages: Iterable[Leaf]) -> bool: def is_root(self, packages: Iterable[Leaf]) -> bool:
""" """
check if package depends on any other package from list of not check if package depends on any other package from list of not
:param packages: list of known leaves
:return: True if any of packages is dependency of the leaf, False otherwise Args:
packages(Iterable[Leaf]): list of known leaves
Returns:
bool: True if any of packages is dependency of the leaf, False otherwise
""" """
for leaf in packages: for leaf in packages:
if self.dependencies.intersection(leaf.items): if self.dependencies.intersection(leaf.items):
@ -84,13 +91,17 @@ class Leaf:
class Tree: class Tree:
""" """
dependency tree implementation dependency tree implementation
:ivar leaves: list of tree leaves
Attributes:
leaves[List[Leaf]): list of tree leaves
""" """
def __init__(self, leaves: List[Leaf]) -> None: def __init__(self, leaves: List[Leaf]) -> None:
""" """
default constructor default constructor
:param leaves: leaves to build the tree
Args:
leaves(List[Leaf]): leaves to build the tree
""" """
self.leaves = leaves self.leaves = leaves
@ -98,16 +109,22 @@ class Tree:
def load(cls: Type[Tree], packages: Iterable[Package], database: SQLite) -> Tree: def load(cls: Type[Tree], packages: Iterable[Package], database: SQLite) -> Tree:
""" """
load tree from packages load tree from packages
:param packages: packages list
:param database: database instance Args:
:return: loaded class packages(Iterable[Package]): packages list
database(SQLite): database instance
Returns:
Tree: loaded class
""" """
return cls([Leaf.load(package, database) for package in packages]) return cls([Leaf.load(package, database) for package in packages])
def levels(self) -> List[List[Package]]: def levels(self) -> List[List[Package]]:
""" """
get build levels starting from the packages which do not require any other package to build get build levels starting from the packages which do not require any other package to build
:return: list of packages lists
Returns:
List[List[Package]]: list of packages lists
""" """
result: List[List[Package]] = [] result: List[List[Package]] = []

View File

@ -32,26 +32,32 @@ from ahriman.models.package import Package
class Github(HttpUpload): class Github(HttpUpload):
""" """
upload files to github releases upload files to github releases
:ivar gh_owner: github repository owner
:ivar gh_repository: github repository name Attributes:
github_owner(str): github repository owner
github_repository(str): github repository name
""" """
def __init__(self, architecture: str, configuration: Configuration, section: str) -> None: def __init__(self, architecture: str, configuration: Configuration, section: str) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture
:param configuration: configuration instance Args:
:param section: settings section name architecture(str): repository architecture
configuration(Configuration): configuration instance
section(str): settings section name
""" """
HttpUpload.__init__(self, architecture, configuration, section) HttpUpload.__init__(self, architecture, configuration, section)
self.gh_owner = configuration.get(section, "owner") self.github_owner = configuration.get(section, "owner")
self.gh_repository = configuration.get(section, "repository") self.github_repository = configuration.get(section, "repository")
def asset_remove(self, release: Dict[str, Any], name: str) -> None: def asset_remove(self, release: Dict[str, Any], name: str) -> None:
""" """
remove asset from the release by name remove asset from the release by name
:param release: release object
:param name: asset name Args:
release(Dict[str, Any]): release object
name(str): asset name
""" """
try: try:
asset = next(asset for asset in release["assets"] if asset["name"] == name) asset = next(asset for asset in release["assets"] if asset["name"] == name)
@ -62,8 +68,10 @@ class Github(HttpUpload):
def asset_upload(self, release: Dict[str, Any], path: Path) -> None: def asset_upload(self, release: Dict[str, Any], path: Path) -> None:
""" """
upload asset to the release upload asset to the release
:param release: release object
:param path: path to local file Args:
release(Dict[str, Any]): release object
path(Path): path to local file
""" """
exists = any(path.name == asset["name"] for asset in release["assets"]) exists = any(path.name == asset["name"] for asset in release["assets"])
if exists: if exists:
@ -76,8 +84,12 @@ class Github(HttpUpload):
def get_local_files(self, path: Path) -> Dict[Path, str]: def get_local_files(self, path: Path) -> Dict[Path, str]:
""" """
get all local files and their calculated checksums get all local files and their calculated checksums
:param path: local path to sync
:return: map of path objects to its checksum Args:
path(Path): local path to sync
Returns:
Dict[Path, str]: map of path objects to its checksum
""" """
return { return {
local_file: self.calculate_hash(local_file) local_file: self.calculate_hash(local_file)
@ -87,9 +99,11 @@ class Github(HttpUpload):
def files_remove(self, release: Dict[str, Any], local_files: Dict[Path, str], remote_files: Dict[str, str]) -> None: def files_remove(self, release: Dict[str, Any], local_files: Dict[Path, str], remote_files: Dict[str, str]) -> None:
""" """
remove files from github remove files from github
:param release: release object
:param local_files: map of local file paths to its checksum Args:
:param remote_files: map of the remote files and its checksum release(Dict[str, Any]): release object
local_files(Dict[Path, str]): map of local file paths to its checksum
remote_files(Dict[str, str]): map of the remote files and its checksum
""" """
local_filenames = {local_file.name for local_file in local_files} local_filenames = {local_file.name for local_file in local_files}
for remote_file in remote_files: for remote_file in remote_files:
@ -100,9 +114,11 @@ class Github(HttpUpload):
def files_upload(self, release: Dict[str, Any], local_files: Dict[Path, str], remote_files: Dict[str, str]) -> None: def files_upload(self, release: Dict[str, Any], local_files: Dict[Path, str], remote_files: Dict[str, str]) -> None:
""" """
upload files to github upload files to github
:param release: release object
:param local_files: map of local file paths to its checksum Args:
:param remote_files: map of the remote files and its checksum release(Dict[str, Any]): release object
local_files(Dict[Path, str]): map of local file paths to its checksum
remote_files(Dict[str, str]): map of the remote files and its checksum
""" """
for local_file, checksum in local_files.items(): for local_file, checksum in local_files.items():
remote_checksum = remote_files.get(local_file.name) remote_checksum = remote_files.get(local_file.name)
@ -113,22 +129,25 @@ class Github(HttpUpload):
def release_create(self) -> Dict[str, Any]: def release_create(self) -> Dict[str, Any]:
""" """
create empty release create empty release
:return: github API release object for the new release
Returns:
Dict[str, Any]: github API release object for the new release
""" """
response = self._request("POST", f"https://api.github.com/repos/{self.gh_owner}/{self.gh_repository}/releases", url = f"https://api.github.com/repos/{self.github_owner}/{self.github_repository}/releases"
json={"tag_name": self.architecture, "name": self.architecture}) response = self._request("POST", url, json={"tag_name": self.architecture, "name": self.architecture})
release: Dict[str, Any] = response.json() release: Dict[str, Any] = response.json()
return release return release
def release_get(self) -> Optional[Dict[str, Any]]: def release_get(self) -> Optional[Dict[str, Any]]:
""" """
get release object if any get release object if any
:return: github API release object if release found and None otherwise
Returns:
Optional[Dict[str, Any]]: github API release object if release found and None otherwise
""" """
url = f"https://api.github.com/repos/{self.github_owner}/{self.github_repository}/releases/tags/{self.architecture}"
try: try:
response = self._request( response = self._request("GET", url)
"GET",
f"https://api.github.com/repos/{self.gh_owner}/{self.gh_repository}/releases/tags/{self.architecture}")
release: Dict[str, Any] = response.json() release: Dict[str, Any] = response.json()
return release return release
except requests.HTTPError as e: except requests.HTTPError as e:
@ -140,16 +159,20 @@ class Github(HttpUpload):
def release_update(self, release: Dict[str, Any], body: str) -> None: def release_update(self, release: Dict[str, Any], body: str) -> None:
""" """
update release update release
:param release: release object
:param body: new release body Args:
release(Dict[str, Any]): release object
body(str): new release body
""" """
self._request("POST", release["url"], json={"body": body}) self._request("POST", release["url"], json={"body": body})
def sync(self, path: Path, built_packages: Iterable[Package]) -> None: def sync(self, path: Path, built_packages: Iterable[Package]) -> None:
""" """
sync data to remote server sync data to remote server
:param path: local path to sync
:param built_packages: list of packages which has just been built Args:
path(Path): local path to sync
built_packages(Iterable[Package]): list of packages which has just been built
""" """
release = self.release_get() release = self.release_get()
if release is None: if release is None:

View File

@ -31,15 +31,19 @@ from ahriman.core.util import exception_response_text
class HttpUpload(Upload): class HttpUpload(Upload):
""" """
helper for the http based uploads helper for the http based uploads
:ivar auth: HTTP auth object
Attributes:
auth(Tuple[str, str]): HTTP auth object
""" """
def __init__(self, architecture: str, configuration: Configuration, section: str) -> None: def __init__(self, architecture: str, configuration: Configuration, section: str) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture
:param configuration: configuration instance Args:
:param section: configuration section name architecture(str): repository architecture
configuration(Configuration): configuration instance
section(str): configuration section name
""" """
Upload.__init__(self, architecture, configuration) Upload.__init__(self, architecture, configuration)
password = configuration.get(section, "password") password = configuration.get(section, "password")
@ -50,8 +54,12 @@ class HttpUpload(Upload):
def calculate_hash(path: Path) -> str: def calculate_hash(path: Path) -> str:
""" """
calculate file checksum calculate file checksum
:param path: path to local file
:return: calculated checksum of the file Args:
path(Path): path to local file
Returns:
str: calculated checksum of the file
""" """
with path.open("rb") as local_file: with path.open("rb") as local_file:
md5 = hashlib.md5(local_file.read()) # nosec md5 = hashlib.md5(local_file.read()) # nosec
@ -61,8 +69,12 @@ class HttpUpload(Upload):
def get_body(local_files: Dict[Path, str]) -> str: def get_body(local_files: Dict[Path, str]) -> str:
""" """
generate release body from the checksums as returned from HttpUpload.get_hashes method generate release body from the checksums as returned from HttpUpload.get_hashes method
:param local_files: map of the paths to its checksum
:return: body to be inserted into release Args:
local_files(Dict[Path, str]): map of the paths to its checksum
Returns:
str: body to be inserted into release
""" """
return "\n".join(f"{file.name} {md5}" for file, md5 in sorted(local_files.items())) return "\n".join(f"{file.name} {md5}" for file, md5 in sorted(local_files.items()))
@ -70,8 +82,12 @@ class HttpUpload(Upload):
def get_hashes(body: str) -> Dict[str, str]: def get_hashes(body: str) -> Dict[str, str]:
""" """
get checksums of the content from the repository get checksums of the content from the repository
:param body: release string body object
:return: map of the filename to its checksum as it is written in body Args:
body(str): release string body object
Returns:
Dict[str, str]: map of the filename to its checksum as it is written in body
""" """
files = {} files = {}
for line in body.splitlines(): for line in body.splitlines():
@ -82,10 +98,14 @@ class HttpUpload(Upload):
def _request(self, method: str, url: str, **kwargs: Any) -> requests.Response: def _request(self, method: str, url: str, **kwargs: Any) -> requests.Response:
""" """
request wrapper request wrapper
:param method: request method
:param url: request url Args:
:param kwargs: request parameters to be passed as is method(str): request method
:return: request response object url(str): request url
**kwargs(Any): request parameters to be passed as is
Returns:
requests.Response: request response object
""" """
try: try:
response = requests.request(method, url, auth=self.auth, **kwargs) response = requests.request(method, url, auth=self.auth, **kwargs)

View File

@ -29,8 +29,10 @@ from ahriman.models.package import Package
class Rsync(Upload): class Rsync(Upload):
""" """
rsync wrapper rsync wrapper
:ivar command: command arguments for sync
:ivar remote: remote address to sync Attributes:
command(List[str]): command arguments for sync
remote(str): remote address to sync
""" """
_check_output = check_output _check_output = check_output
@ -38,9 +40,11 @@ class Rsync(Upload):
def __init__(self, architecture: str, configuration: Configuration, section: str) -> None: def __init__(self, architecture: str, configuration: Configuration, section: str) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture
:param configuration: configuration instance Args:
:param section: settings section name architecture(str): repository architecture
configuration(Configuration): configuration instance
section(str): settings section name
""" """
Upload.__init__(self, architecture, configuration) Upload.__init__(self, architecture, configuration)
self.command = configuration.getlist(section, "command") self.command = configuration.getlist(section, "command")
@ -49,7 +53,9 @@ class Rsync(Upload):
def sync(self, path: Path, built_packages: Iterable[Package]) -> None: def sync(self, path: Path, built_packages: Iterable[Package]) -> None:
""" """
sync data to remote server sync data to remote server
:param path: local path to sync
:param built_packages: list of packages which has just been built Args:
path(Path): local path to sync
built_packages(Iterable[Package]): list of packages which has just been built
""" """
Rsync._check_output(*self.command, str(path), self.remote, exception=None, logger=self.logger) Rsync._check_output(*self.command, str(path), self.remote, exception=None, logger=self.logger)

View File

@ -32,16 +32,21 @@ from ahriman.models.package import Package
class S3(Upload): class S3(Upload):
""" """
aws-cli wrapper boto3 wrapper
:ivar bucket: boto3 S3 bucket object
:ivar chunk_size: chunk size for calculating checksums Attributes
bucket(Any): boto3 S3 bucket object
chunk_size(int): chunk size for calculating checksums
""" """
def __init__(self, architecture: str, configuration: Configuration, section: str) -> None: def __init__(self, architecture: str, configuration: Configuration, section: str) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture
:param configuration: configuration instance Args:
architecture(str): repository architecture
configuration(Configuration): configuration instance
section(str): settings section name
""" """
Upload.__init__(self, architecture, configuration) Upload.__init__(self, architecture, configuration)
self.bucket = self.get_bucket(configuration, section) self.bucket = self.get_bucket(configuration, section)
@ -53,9 +58,13 @@ class S3(Upload):
calculate amazon s3 etag calculate amazon s3 etag
credits to https://teppen.io/2018/10/23/aws_s3_verify_etags/ credits to https://teppen.io/2018/10/23/aws_s3_verify_etags/
For this method we have to define nosec because it is out of any security context and provided by AWS For this method we have to define nosec because it is out of any security context and provided by AWS
:param path: path to local file
:param chunk_size: read chunk size, which depends on client settings Args:
:return: calculated entity tag for local file path(Path): path to local file
chunk_size(int): read chunk size, which depends on client settings
Returns:
str: calculated entity tag for local file
""" """
md5s = [] md5s = []
with path.open("rb") as local_file: with path.open("rb") as local_file:
@ -73,9 +82,13 @@ class S3(Upload):
def get_bucket(configuration: Configuration, section: str) -> Any: def get_bucket(configuration: Configuration, section: str) -> Any:
""" """
create resource client from configuration create resource client from configuration
:param configuration: configuration instance
:param section: settings section name Args:
:return: amazon client configuration(Configuration): configuration instance
section(str): settings section name
Returns:
Any: amazon client
""" """
client = boto3.resource(service_name="s3", client = boto3.resource(service_name="s3",
region_name=configuration.get(section, "region"), region_name=configuration.get(section, "region"),
@ -87,8 +100,10 @@ class S3(Upload):
def files_remove(local_files: Dict[Path, str], remote_objects: Dict[Path, Any]) -> None: def files_remove(local_files: Dict[Path, str], remote_objects: Dict[Path, Any]) -> None:
""" """
remove files which have been removed locally remove files which have been removed locally
:param local_files: map of local path object to its checksum
:param remote_objects: map of remote path object to the remote s3 object Args:
local_files(Dict[Path, str]): map of local path object to its checksum
remote_objects(Dict[Path, Any]): map of remote path object to the remote s3 object
""" """
for local_file, remote_object in remote_objects.items(): for local_file, remote_object in remote_objects.items():
if local_file in local_files: if local_file in local_files:
@ -98,9 +113,11 @@ class S3(Upload):
def files_upload(self, path: Path, local_files: Dict[Path, str], remote_objects: Dict[Path, Any]) -> None: def files_upload(self, path: Path, local_files: Dict[Path, str], remote_objects: Dict[Path, Any]) -> None:
""" """
upload changed files to s3 upload changed files to s3
:param path: local path to sync
:param local_files: map of local path object to its checksum Args:
:param remote_objects: map of remote path object to the remote s3 object path(Path): local path to sync
local_files(Dict[Path, str]): map of local path object to its checksum
remote_objects(Dict[Path, Any]): map of remote path object to the remote s3 object
""" """
for local_file, checksum in local_files.items(): for local_file, checksum in local_files.items():
remote_object = remote_objects.get(local_file) remote_object = remote_objects.get(local_file)
@ -119,8 +136,12 @@ class S3(Upload):
def get_local_files(self, path: Path) -> Dict[Path, str]: def get_local_files(self, path: Path) -> Dict[Path, str]:
""" """
get all local files and their calculated checksums get all local files and their calculated checksums
:param path: local path to sync
:return: map of path object to its checksum Args:
path(Path): local path to sync
Returns:
Dict[Path, str]: map of path object to its checksum
""" """
return { return {
local_file.relative_to(path): self.calculate_etag(local_file, self.chunk_size) local_file.relative_to(path): self.calculate_etag(local_file, self.chunk_size)
@ -130,7 +151,9 @@ class S3(Upload):
def get_remote_objects(self) -> Dict[Path, Any]: def get_remote_objects(self) -> Dict[Path, Any]:
""" """
get all remote objects and their checksums get all remote objects and their checksums
:return: map of path object to the remote s3 object
Returns:
Dict[Path, Any]: map of path object to the remote s3 object
""" """
objects = self.bucket.objects.filter(Prefix=self.architecture) objects = self.bucket.objects.filter(Prefix=self.architecture)
return {Path(item.key).relative_to(self.architecture): item for item in objects} return {Path(item.key).relative_to(self.architecture): item for item in objects}
@ -138,8 +161,10 @@ class S3(Upload):
def sync(self, path: Path, built_packages: Iterable[Package]) -> None: def sync(self, path: Path, built_packages: Iterable[Package]) -> None:
""" """
sync data to remote server sync data to remote server
:param path: local path to sync
:param built_packages: list of packages which has just been built Args:
path(Path): local path to sync
built_packages(Iterable[Package]): list of packages which has just been built
""" """
remote_objects = self.get_remote_objects() remote_objects = self.get_remote_objects()
local_files = self.get_local_files(path) local_files = self.get_local_files(path)

View File

@ -33,16 +33,20 @@ from ahriman.models.upload_settings import UploadSettings
class Upload: class Upload:
""" """
base remote sync class base remote sync class
:ivar architecture: repository architecture
:ivar configuration: configuration instance Attributes:
:ivar logger: application logger architecture(str): repository architecture
configuration(Configuration): configuration instance
logger(logging.Logger): application logger
""" """
def __init__(self, architecture: str, configuration: Configuration) -> None: def __init__(self, architecture: str, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture
:param configuration: configuration instance Args:
architecture(str): repository architecture
configuration(Configuration): configuration instance
""" """
self.logger = logging.getLogger("root") self.logger = logging.getLogger("root")
self.architecture = architecture self.architecture = architecture
@ -52,10 +56,14 @@ class Upload:
def load(cls: Type[Upload], architecture: str, configuration: Configuration, target: str) -> Upload: def load(cls: Type[Upload], architecture: str, configuration: Configuration, target: str) -> Upload:
""" """
load client from settings load client from settings
:param architecture: repository architecture
:param configuration: configuration instance Args:
:param target: target to run sync (e.g. s3) architecture(str): repository architecture
:return: client according to current settings configuration(Configuration): configuration instance
target(str): target to run sync (e.g. s3)
Returns:
Upload: client according to current settings
""" """
section, provider_name = configuration.gettype(target, architecture) section, provider_name = configuration.gettype(target, architecture)
provider = UploadSettings.from_option(provider_name) provider = UploadSettings.from_option(provider_name)
@ -73,8 +81,13 @@ class Upload:
def run(self, path: Path, built_packages: Iterable[Package]) -> None: def run(self, path: Path, built_packages: Iterable[Package]) -> None:
""" """
run remote sync run remote sync
:param path: local path to sync
:param built_packages: list of packages which has just been built Args:
path(Path): local path to sync
built_packages(Iterable[Package]): list of packages which has just been built
Raises:
SyncFailed: in case of any synchronization unmatched exception
""" """
try: try:
self.sync(path, built_packages) self.sync(path, built_packages)
@ -85,6 +98,8 @@ class Upload:
def sync(self, path: Path, built_packages: Iterable[Package]) -> None: def sync(self, path: Path, built_packages: Iterable[Package]) -> None:
""" """
sync data to remote server sync data to remote server
:param path: local path to sync
:param built_packages: list of packages which has just been built Args:
path(Path): local path to sync
built_packages(Iterable[Package]): list of packages which has just been built
""" """

View File

@ -37,13 +37,20 @@ def check_output(*args: str, exception: Optional[Exception], cwd: Optional[Path]
input_data: Optional[str] = None, logger: Optional[Logger] = None, user: Optional[int] = None) -> str: input_data: Optional[str] = None, logger: Optional[Logger] = None, user: Optional[int] = None) -> str:
""" """
subprocess wrapper subprocess wrapper
:param args: command line arguments
:param exception: exception which has to be reraised instead of default subprocess exception Args:
:param cwd: current working directory *args(str): command line arguments
:param input_data: data which will be written to command stdin exception(Optional[Exception]): exception which has to be reraised instead of default subprocess exception
:param logger: logger to log command result if required cwd(Optional[Path], optional): current working directory (Default value = None)
:param user: run process as specified user input_data(Optional[str], optional): data which will be written to command stdin (Default value = None)
:return: command output logger(Optional[Logger], optional): logger to log command result if required (Default value = None)
user(Optional[int], optional): run process as specified user (Default value = None)
Returns:
str: command output
Raises:
subprocess.CalledProcessError: if subprocess ended with status code different from 0 and no exception supplied
""" """
def log(single: str) -> None: def log(single: str) -> None:
if logger is not None: if logger is not None:
@ -83,8 +90,13 @@ def check_output(*args: str, exception: Optional[Exception], cwd: Optional[Path]
def check_user(paths: RepositoryPaths, unsafe: bool) -> None: def check_user(paths: RepositoryPaths, unsafe: bool) -> None:
""" """
check if current user is the owner of the root check if current user is the owner of the root
:param paths: repository paths object
:param unsafe: if set no user check will be performed before path creation Args:
paths(RepositoryPaths): repository paths object
unsafe(bool): if set no user check will be performed before path creation
Raises:
UnsafeRun: if root uid differs from current uid and check is enabled
""" """
if not paths.root.exists(): if not paths.root.exists():
return # no directory found, skip check return # no directory found, skip check
@ -99,8 +111,12 @@ def check_user(paths: RepositoryPaths, unsafe: bool) -> None:
def exception_response_text(exception: requests.exceptions.HTTPError) -> str: def exception_response_text(exception: requests.exceptions.HTTPError) -> str:
""" """
safe response exception text generation safe response exception text generation
:param exception: exception raised
:return: text of the response if it is not None and empty string otherwise Args:
exception(requests.exceptions.HTTPError): exception raised
Returns:
str: text of the response if it is not None and empty string otherwise
""" """
result: str = exception.response.text if exception.response is not None else "" result: str = exception.response.text if exception.response is not None else ""
return result return result
@ -109,18 +125,42 @@ def exception_response_text(exception: requests.exceptions.HTTPError) -> str:
def filter_json(source: Dict[str, Any], known_fields: Iterable[str]) -> Dict[str, Any]: def filter_json(source: Dict[str, Any], known_fields: Iterable[str]) -> Dict[str, Any]:
""" """
filter json object by fields used for json-to-object conversion filter json object by fields used for json-to-object conversion
:param source: raw json object
:param known_fields: list of fields which have to be known for the target object Args:
:return: json object without unknown and empty fields source(Dict[str, Any]): raw json object
known_fields(Iterable[str]): list of fields which have to be known for the target object
Returns:
Dict[str, Any]: json object without unknown and empty fields
""" """
return {key: value for key, value in source.items() if key in known_fields and value is not None} return {key: value for key, value in source.items() if key in known_fields and value is not None}
def full_version(epoch: Union[str, int, None], pkgver: str, pkgrel: str) -> str:
"""
generate full version from components
Args:
epoch(Union[str, int, None]): package epoch if any
pkgver(str): package version
pkgrel(str): package release version (arch linux specific)
Returns:
str: generated version
"""
prefix = f"{epoch}:" if epoch else ""
return f"{prefix}{pkgver}-{pkgrel}"
def package_like(filename: Path) -> bool: def package_like(filename: Path) -> bool:
""" """
check if file looks like package check if file looks like package
:param filename: name of file to check
:return: True in case if name contains `.pkg.` and not signature, False otherwise Args:
filename(Path): name of file to check
Returns:
bool: True in case if name contains `.pkg.` and not signature, False otherwise
""" """
name = filename.name name = filename.name
return ".pkg." in name and not name.endswith(".sig") return ".pkg." in name and not name.endswith(".sig")
@ -129,8 +169,12 @@ def package_like(filename: Path) -> bool:
def pretty_datetime(timestamp: Optional[Union[datetime.datetime, float, int]]) -> str: def pretty_datetime(timestamp: Optional[Union[datetime.datetime, float, int]]) -> str:
""" """
convert datetime object to string convert datetime object to string
:param timestamp: datetime to convert
:return: pretty printable datetime as string Args:
timestamp(Optional[Union[datetime.datetime, float, int]]): datetime to convert
Returns:
str: pretty printable datetime as string
""" """
if timestamp is None: if timestamp is None:
return "" return ""
@ -142,9 +186,16 @@ def pretty_datetime(timestamp: Optional[Union[datetime.datetime, float, int]]) -
def pretty_size(size: Optional[float], level: int = 0) -> str: def pretty_size(size: Optional[float], level: int = 0) -> str:
""" """
convert size to string convert size to string
:param size: size to convert
:param level: represents current units, 0 is B, 1 is KiB etc Args:
:return: pretty printable size as string size(Optional[float]): size to convert
level(int, optional): represents current units, 0 is B, 1 is KiB etc (Default value = 0)
Returns:
str: pretty printable size as string
Raises:
InvalidOption: if size is more than 1TiB
""" """
def str_level() -> str: def str_level() -> str:
if level == 0: if level == 0:
@ -168,7 +219,9 @@ def pretty_size(size: Optional[float], level: int = 0) -> str:
def tmpdir() -> Generator[Path, None, None]: def tmpdir() -> Generator[Path, None, None]:
""" """
wrapper for tempfile to remove directory after all wrapper for tempfile to remove directory after all
:return: path to the created directory
Yields:
Path: path to the created directory
""" """
path = Path(tempfile.mkdtemp()) path = Path(tempfile.mkdtemp())
try: try:
@ -181,8 +234,12 @@ def walk(directory_path: Path) -> Generator[Path, None, None]:
""" """
list all file paths in given directory list all file paths in given directory
Credits to https://stackoverflow.com/a/64915960 Credits to https://stackoverflow.com/a/64915960
:param directory_path: root directory path
:return: all found files in given directory with full path Args:
directory_path(Path): root directory path
Yields:
Path: all found files in given directory with full path
""" """
for element in directory_path.iterdir(): for element in directory_path.iterdir():
if element.is_dir(): if element.is_dir():

View File

@ -23,9 +23,11 @@ from enum import Enum
class Action(Enum): class Action(Enum):
""" """
base action enumeration base action enumeration
:cvar List: list available values
:cvar Remove: remove everything from local storage Attributes:
:cvar Update: update local storage or add to List(Action): (class attribute) list available values
Remove(Action): (class attribute) remove everything from local storage
Update(Action): (class attribute) update local storage or add to
""" """
List = "list" List = "list"

Some files were not shown because too many files have changed in this diff Show More