Compare commits

...

19 Commits

Author SHA1 Message Date
32cbafd12b Release 0.20.0 2021-04-01 02:38:59 +03:00
880c70bd58 constistent classmethod and staticmethod usage
General idea is to use classmethod for every constructor and
statismethod otherwise.
Also use self and cls whenever it's possible to call static and class
methods
2021-03-31 04:29:08 +03:00
d449eb3c2e change arch specific section naming from section_arch to section:arch
Some archs can have _ in their name. Also in future we can use sections
with similar names
2021-03-31 02:31:14 +03:00
17b5cd0751 Release 0.19.0 2021-03-31 02:19:44 +03:00
2aef906fc8 add now argument to add command 2021-03-31 02:19:13 +03:00
e034327501 filter out every foreign arch in config 2021-03-31 01:55:39 +03:00
5d79fcca22 read sign targets from correct path 2021-03-31 00:11:05 +03:00
6e9dcca254 make configuration object arch-specific 2021-03-31 00:04:13 +03:00
fbf6748d4a more verbose variables 2021-03-30 05:29:13 +03:00
2260e52d5c merge settings groups instead of using whole group 2021-03-30 04:58:15 +03:00
bd2b61494f move rsync and s3 options to configuration 2021-03-30 02:38:18 +03:00
7280d30748 verbose help message 2021-03-30 02:25:23 +03:00
710274065d add status badge 2021-03-30 02:01:59 +03:00
e0b09cefad rename gpg test to correct naming 2021-03-30 01:53:23 +03:00
3b93510aad add more tests 2021-03-30 01:42:01 +03:00
5003cabeb5 Release 0.18.0 2021-03-29 11:48:54 +03:00
bc6af9256b more properties to be shown in status pages 2021-03-29 11:48:32 +03:00
1ac7c87317 architecture depending pacman.conf 2021-03-29 10:08:11 +03:00
803b7bee1e add status update subcommand
also satisfy pylint with too big method with too much variables
2021-03-29 04:17:10 +03:00
75 changed files with 1437 additions and 577 deletions

View File

@ -1,6 +1,6 @@
# ahriman configuration # ahriman configuration
Some groups can be specified for each architecture separately. E.g. if there are `build` and `build_x86_64` groups it will use the `build_x86_64` for the `x86_64` architecture and `build` for any other (architecture specific group has higher priority). Some groups can be specified for each architecture separately. E.g. if there are `build` and `build:x86_64` groups it will use the option from `build:x86_64` for the `x86_64` architecture and `build` for any other (architecture specific group has higher priority). In case if both groups are presented, architecture specific options will be merged into global ones overriding them.
## `settings` group ## `settings` group
@ -18,9 +18,9 @@ libalpm and AUR related configuration.
* `repositories` - list of pacman repositories, space separated list of strings, required. * `repositories` - list of pacman repositories, space separated list of strings, required.
* `root` - root for alpm library, string, required. * `root` - root for alpm library, string, required.
## `build_*` groups ## `build:*` groups
Build related configuration. Group name must refer to architecture, e.g. it should be `build_x86_64` for x86_64 architecture. Build related configuration. Group name must refer to architecture, e.g. it should be `build:x86_64` for x86_64 architecture.
* `archbuild_flags` - additional flags passed to `archbuild` command, space separated list of strings, optional. * `archbuild_flags` - additional flags passed to `archbuild` command, space separated list of strings, optional.
* `build_command` - default build command, string, required. * `build_command` - default build command, string, required.
@ -35,9 +35,9 @@ Base repository settings.
* `name` - repository name, string, required. * `name` - repository name, string, required.
* `root` - root path for application, string, required. * `root` - root path for application, string, required.
## `sign_*` groups ## `sign:*` groups
Settings for signing packages or repository. Group name must refer to architecture, e.g. it should be `sign_x86_64` for x86_64 architecture. Settings for signing packages or repository. Group name must refer to architecture, e.g. it should be `sign:x86_64` for x86_64 architecture.
* `target` - configuration flag to enable signing, space separated list of strings, required. Allowed values are `package` (sign each package separately), `repository` (sign repository database file). * `target` - configuration flag to enable signing, space separated list of strings, required. Allowed values are `package` (sign each package separately), `repository` (sign repository database file).
* `key` - default PGP key, string, required. This key will also be used for database signing if enabled. * `key` - default PGP key, string, required. This key will also be used for database signing if enabled.
@ -49,9 +49,9 @@ Report generation settings.
* `target` - list of reports to be generated, space separated list of strings, optional. Allowed values are `html`. * `target` - list of reports to be generated, space separated list of strings, optional. Allowed values are `html`.
### `html_*` groups ### `html:*` groups
Group name must refer to architecture, e.g. it should be `html_x86_64` for x86_64 architecture. Group name must refer to architecture, e.g. it should be `html:x86_64` for x86_64 architecture.
* `path` - path to html report file, string, required. * `path` - path to html report file, string, required.
* `homepage` - link to homepage, string, optional. * `homepage` - link to homepage, string, optional.
@ -64,21 +64,23 @@ Remote synchronization settings.
* `target` - list of synchronizations to be used, space separated list of strings, optional. Allowed values are `rsync`, `s3`. * `target` - list of synchronizations to be used, space separated list of strings, optional. Allowed values are `rsync`, `s3`.
### `rsync_*` groups ### `rsync:*` groups
Group name must refer to architecture, e.g. it should be `rsync_x86_64` for x86_64 architecture. Requires `rsync` package to be installed. Do not forget to configure ssh for user `ahriman`. Group name must refer to architecture, e.g. it should be `rsync:x86_64` for x86_64 architecture. Requires `rsync` package to be installed. Do not forget to configure ssh for user `ahriman`.
* `command` - rsync command to run, space separated list of string, required.
* `remote` - remote server to rsync (e.g. `1.2.3.4:5678:path/to/sync`), string, required. * `remote` - remote server to rsync (e.g. `1.2.3.4:5678:path/to/sync`), string, required.
### `s3_*` groups ### `s3:*` groups
Group name must refer to architecture, e.g. it should be `s3_x86_64` for x86_64 architecture. Requires `aws-cli` package to be installed. Do not forget to configure it for user `ahriman`. Group name must refer to architecture, e.g. it should be `s3:x86_64` for x86_64 architecture. Requires `aws-cli` package to be installed. Do not forget to configure it for user `ahriman`.
* `command` - s3 command to run, space separated list of string, required.
* `bucket` - bucket name (e.g. `s3://bucket/path`), string, required. * `bucket` - bucket name (e.g. `s3://bucket/path`), string, required.
## `web_*` groups ## `web:*` groups
Web server settings. If any of `host`/`port` is not set, web integration will be disabled. Group name must refer to architecture, e.g. it should be `web_x86_64` for x86_64 architecture. Web server settings. If any of `host`/`port` is not set, web integration will be disabled. Group name must refer to architecture, e.g. it should be `web:x86_64` for x86_64 architecture.
* `host` - host to bind, string, optional. * `host` - host to bind, string, optional.
* `port` - port to bind, int, optional. * `port` - port to bind, int, optional.

View File

@ -24,7 +24,7 @@ archlinux: archive
sed -i "/sha512sums=('[0-9A-Fa-f]*/s/[^'][^)]*/sha512sums=('$$(sha512sum $(PROJECT)-$(VERSION)-src.tar.xz | awk '{print $$1}')'/" package/archlinux/PKGBUILD sed -i "/sha512sums=('[0-9A-Fa-f]*/s/[^'][^)]*/sha512sums=('$$(sha512sum $(PROJECT)-$(VERSION)-src.tar.xz | awk '{print $$1}')'/" package/archlinux/PKGBUILD
sed -i "s/pkgver=[0-9.]*/pkgver=$(VERSION)/" package/archlinux/PKGBUILD sed -i "s/pkgver=[0-9.]*/pkgver=$(VERSION)/" package/archlinux/PKGBUILD
check: check: clean
cd src && mypy --implicit-reexport --strict -p "$(PROJECT)" cd src && mypy --implicit-reexport --strict -p "$(PROJECT)"
find "src/$(PROJECT)" tests -name "*.py" -execdir autopep8 --exit-code --max-line-length 120 -aa -i {} + find "src/$(PROJECT)" tests -name "*.py" -execdir autopep8 --exit-code --max-line-length 120 -aa -i {} +
cd src && pylint --rcfile=../.pylintrc "$(PROJECT)" cd src && pylint --rcfile=../.pylintrc "$(PROJECT)"
@ -43,7 +43,7 @@ push: archlinux
git tag "$(VERSION)" git tag "$(VERSION)"
git push --tags git push --tags
tests: tests: clean
python setup.py test python setup.py test
version: version:

View File

@ -1,5 +1,7 @@
# ArcHlinux ReposItory MANager # ArcHlinux ReposItory MANager
![build status](https://github.com/arcan1s/ahriman/actions/workflows/python-app.yml/badge.svg)
Wrapper for managing custom repository inspired by [repo-scripts](https://github.com/arcan1s/repo-scripts). Wrapper for managing custom repository inspired by [repo-scripts](https://github.com/arcan1s/repo-scripts).
## Features ## Features
@ -64,7 +66,7 @@ Wrapper for managing custom repository inspired by [repo-scripts](https://github
* Add packages by using `ahriman add {package}` command: * Add packages by using `ahriman add {package}` command:
```shell ```shell
sudo -u ahriman ahriman -a x86_64 add yay sudo -u ahriman ahriman -a x86_64 add yay --now
``` ```
Note that initial service configuration can be done by running `ahriman setup` with specific arguments. Note that initial service configuration can be done by running `ahriman setup` with specific arguments.

View File

@ -1,7 +1,7 @@
# Maintainer: Evgeniy Alekseev # Maintainer: Evgeniy Alekseev
pkgname='ahriman' pkgname='ahriman'
pkgver=0.17.0 pkgver=0.20.0
pkgrel=1 pkgrel=1
pkgdesc="ArcHlinux ReposItory MANager" pkgdesc="ArcHlinux ReposItory MANager"
arch=('any') arch=('any')
@ -23,7 +23,7 @@ optdepends=('aws-cli: sync to s3'
source=("https://github.com/arcan1s/ahriman/releases/download/$pkgver/$pkgname-$pkgver-src.tar.xz" source=("https://github.com/arcan1s/ahriman/releases/download/$pkgver/$pkgname-$pkgver-src.tar.xz"
'ahriman.sysusers' 'ahriman.sysusers'
'ahriman.tmpfiles') 'ahriman.tmpfiles')
sha512sums=('0e29a7a075cbca0259a2f6374a5d1d09285112f6221d62b1c24b56ad23807133001d56ed045307866f9c7cde772f54adda234bdfcaaa272ac7d363c7da6d1f9f' sha512sums=('22d2d2ae5af4a5854eb08b3b97a5fe4faa85246a85ffa78fa350080cf40a42b77152a83e0184ba848a8d06a0cce8a02a6fd94e24982ebbcc80bb88901d229f25'
'13718afec2c6786a18f0b223ef8e58dccf0688bca4cdbe203f14071f5031ed20120eb0ce38b52c76cfd6e8b6581a9c9eaa2743eb11abbaca637451a84c33f075' '13718afec2c6786a18f0b223ef8e58dccf0688bca4cdbe203f14071f5031ed20120eb0ce38b52c76cfd6e8b6581a9c9eaa2743eb11abbaca637451a84c33f075'
'55b20f6da3d66e7bbf2add5d95a3b60632df121717d25a993e56e737d14f51fe063eb6f1b38bd81cc32e05db01c0c1d80aaa720c45cde87f238d8b46cdb8cbc4') '55b20f6da3d66e7bbf2add5d95a3b60632df121717d25a993e56e737d14f51fe063eb6f1b38bd81cc32e05db01c0c1d80aaa720c45cde87f238d8b46cdb8cbc4')
backup=('etc/ahriman.ini' backup=('etc/ahriman.ini'

View File

@ -21,7 +21,6 @@ root = /var/lib/ahriman
[sign] [sign]
target = target =
key =
[report] [report]
target = target =
@ -36,10 +35,10 @@ template_path = /usr/share/ahriman/repo-index.jinja2
target = target =
[rsync] [rsync]
remote = command = rsync --archive --verbose --compress --partial --delete
[s3] [s3]
bucket = command = aws s3 sync --quiet --delete
[web] [web]
templates = /usr/share/ahriman templates = /usr/share/ahriman

View File

@ -23,17 +23,24 @@ import sys
import ahriman.application.handlers as handlers import ahriman.application.handlers as handlers
import ahriman.version as version import ahriman.version as version
from ahriman.models.build_status import BuildStatusEnum
# pylint thinks it is bad idea, but get the fuck off
# pylint: disable=protected-access
SubParserAction = argparse._SubParsersAction
# pylint: disable=too-many-statements
def _parser() -> argparse.ArgumentParser: def _parser() -> argparse.ArgumentParser:
""" """
command line parser generator command line parser generator
:return: command line parser for the application :return: command line parser for the application
""" """
parser = argparse.ArgumentParser(prog="ahriman", description="ArcHlinux ReposItory MANager") parser = argparse.ArgumentParser(prog="ahriman", description="ArcHlinux ReposItory MANager",
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument("-a", "--architecture", help="target architectures (can be used multiple times)", parser.add_argument("-a", "--architecture", help="target architectures (can be used multiple times)",
action="append", required=True) action="append", required=True)
parser.add_argument("-c", "--config", help="configuration path", default="/etc/ahriman.ini") parser.add_argument("-c", "--configuration", help="configuration path", default="/etc/ahriman.ini")
parser.add_argument("--force", help="force run, remove file lock", action="store_true") parser.add_argument("--force", help="force run, remove file lock", action="store_true")
parser.add_argument("--lock", help="lock file", default="/tmp/ahriman.lock") parser.add_argument("--lock", help="lock file", default="/tmp/ahriman.lock")
parser.add_argument("--no-log", help="redirect all log messages to stderr", action="store_true") parser.add_argument("--no-log", help="redirect all log messages to stderr", action="store_true")
@ -43,73 +50,225 @@ def _parser() -> argparse.ArgumentParser:
subparsers = parser.add_subparsers(title="command", help="command to run", dest="command", required=True) subparsers = parser.add_subparsers(title="command", help="command to run", dest="command", required=True)
add_parser = subparsers.add_parser("add", description="add package") _set_add_parser(subparsers)
add_parser.add_argument("package", help="package base/name or archive path", nargs="+") _set_check_parser(subparsers)
add_parser.add_argument("--without-dependencies", help="do not add dependencies", action="store_true") _set_clean_parser(subparsers)
add_parser.set_defaults(handler=handlers.Add) _set_config_parser(subparsers)
_set_rebuild_parser(subparsers)
_set_remove_parser(subparsers)
_set_report_parser(subparsers)
_set_setup_parser(subparsers)
_set_sign_parser(subparsers)
_set_status_parser(subparsers)
_set_status_update_parser(subparsers)
_set_sync_parser(subparsers)
_set_update_parser(subparsers)
_set_web_parser(subparsers)
check_parser = subparsers.add_parser("check", description="check for updates. Same as update --dry-run --no-manual") return parser
check_parser.add_argument("package", help="filter check by package base", nargs="*")
check_parser.add_argument("--no-vcs", help="do not check VCS packages", action="store_true")
check_parser.set_defaults(handler=handlers.Update, no_aur=False, no_manual=True, dry_run=True)
clean_parser = subparsers.add_parser("clean", description="clear all local caches")
clean_parser.add_argument("--no-build", help="do not clear directory with package sources", action="store_true")
clean_parser.add_argument("--no-cache", help="do not clear directory with package caches", action="store_true")
clean_parser.add_argument("--no-chroot", help="do not clear build chroot", action="store_true")
clean_parser.add_argument("--no-manual", help="do not clear directory with manually added packages",
action="store_true")
clean_parser.add_argument("--no-packages", help="do not clear directory with built packages", action="store_true")
clean_parser.set_defaults(handler=handlers.Clean, unsafe=True)
config_parser = subparsers.add_parser("config", description="dump configuration for specified architecture") def _set_add_parser(root: SubParserAction) -> argparse.ArgumentParser:
config_parser.set_defaults(handler=handlers.Dump, lock=None, no_report=True, unsafe=True) """
add parser for add subcommand
:param root: subparsers for the commands
:return: created argument parser
"""
parser = root.add_parser("add", help="add package", description="add package",
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument("package", help="package base/name or archive path", nargs="+")
parser.add_argument("--now", help="run update function after", action="store_true")
parser.add_argument("--without-dependencies", help="do not add dependencies", action="store_true")
parser.set_defaults(handler=handlers.Add)
return parser
rebuild_parser = subparsers.add_parser("rebuild", description="rebuild whole repository")
rebuild_parser.set_defaults(handler=handlers.Rebuild)
remove_parser = subparsers.add_parser("remove", description="remove package") def _set_check_parser(root: SubParserAction) -> argparse.ArgumentParser:
remove_parser.add_argument("package", help="package name or base", nargs="+") """
remove_parser.set_defaults(handler=handlers.Remove) add parser for check subcommand
:param root: subparsers for the commands
:return: created argument parser
"""
parser = root.add_parser("check", help="check for updates",
description="check for updates. Same as update --dry-run --no-manual",
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument("package", help="filter check by package base", nargs="*")
parser.add_argument("--no-vcs", help="do not check VCS packages", action="store_true")
parser.set_defaults(handler=handlers.Update, no_aur=False, no_manual=True, dry_run=True)
return parser
report_parser = subparsers.add_parser("report", description="generate report")
report_parser.add_argument("target", help="target to generate report", nargs="*")
report_parser.set_defaults(handler=handlers.Report)
setup_parser = subparsers.add_parser("setup", description="create initial service configuration, requires root") def _set_clean_parser(root: SubParserAction) -> argparse.ArgumentParser:
setup_parser.add_argument("--build-command", help="build command prefix", default="ahriman") """
setup_parser.add_argument("--from-config", help="path to default devtools pacman configuration", add parser for clean subcommand
default="/usr/share/devtools/pacman-extra.conf") :param root: subparsers for the commands
setup_parser.add_argument("--no-multilib", help="do not add multilib repository", action="store_true") :return: created argument parser
setup_parser.add_argument("--packager", help="packager name and email", required=True) """
setup_parser.add_argument("--repository", help="repository name", default="aur-clone") parser = root.add_parser("clean", help="clean local caches", description="clear local caches",
setup_parser.set_defaults(handler=handlers.Setup, lock=None, no_report=True, unsafe=True) formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument("--no-build", help="do not clear directory with package sources", action="store_true")
parser.add_argument("--no-cache", help="do not clear directory with package caches", action="store_true")
parser.add_argument("--no-chroot", help="do not clear build chroot", action="store_true")
parser.add_argument("--no-manual", help="do not clear directory with manually added packages", action="store_true")
parser.add_argument("--no-packages", help="do not clear directory with built packages", action="store_true")
parser.set_defaults(handler=handlers.Clean, unsafe=True)
return parser
sign_parser = subparsers.add_parser("sign", description="(re-)sign packages and repository database")
sign_parser.add_argument("package", help="sign only specified packages", nargs="*")
sign_parser.set_defaults(handler=handlers.Sign)
status_parser = subparsers.add_parser("status", description="request status of the package") def _set_config_parser(root: SubParserAction) -> argparse.ArgumentParser:
status_parser.add_argument("--ahriman", help="get service status itself", action="store_true") """
status_parser.add_argument("package", help="filter status by package base", nargs="*") add parser for config subcommand
status_parser.set_defaults(handler=handlers.Status, lock=None, no_report=True, unsafe=True) :param root: subparsers for the commands
:return: created argument parser
"""
parser = root.add_parser("config", help="dump configuration",
description="dump configuration for specified architecture",
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.set_defaults(handler=handlers.Dump, lock=None, no_report=True, unsafe=True)
return parser
sync_parser = subparsers.add_parser("sync", description="sync packages to remote server")
sync_parser.add_argument("target", help="target to sync", nargs="*")
sync_parser.set_defaults(handler=handlers.Sync)
update_parser = subparsers.add_parser("update", description="run updates") def _set_rebuild_parser(root: SubParserAction) -> argparse.ArgumentParser:
update_parser.add_argument("package", help="filter check by package base", nargs="*") """
update_parser.add_argument("--dry-run", help="just perform check for updates, same as check command", add parser for rebuild subcommand
action="store_true") :param root: subparsers for the commands
update_parser.add_argument("--no-aur", help="do not check for AUR updates. Implies --no-vcs", action="store_true") :return: created argument parser
update_parser.add_argument("--no-manual", help="do not include manual updates", action="store_true") """
update_parser.add_argument("--no-vcs", help="do not check VCS packages", action="store_true") parser = root.add_parser("rebuild", help="rebuild repository", description="rebuild whole repository",
update_parser.set_defaults(handler=handlers.Update) formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.set_defaults(handler=handlers.Rebuild)
return parser
web_parser = subparsers.add_parser("web", description="start web server")
web_parser.set_defaults(handler=handlers.Web, lock=None, no_report=True)
def _set_remove_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for remove subcommand
:param root: subparsers for the commands
:return: created argument parser
"""
parser = root.add_parser("remove", help="remove package", description="remove package",
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument("package", help="package name or base", nargs="+")
parser.set_defaults(handler=handlers.Remove)
return parser
def _set_report_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for report subcommand
:param root: subparsers for the commands
:return: created argument parser
"""
parser = root.add_parser("report", help="generate report", description="generate report",
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument("target", help="target to generate report", nargs="*")
parser.set_defaults(handler=handlers.Report)
return parser
def _set_setup_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for setup subcommand
:param root: subparsers for the commands
:return: created argument parser
"""
parser = root.add_parser("setup", help="initial service configuration",
description="create initial service configuration, requires root",
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument("--build-command", help="build command prefix", default="ahriman")
parser.add_argument("--from-configuration", help="path to default devtools pacman configuration",
default="/usr/share/devtools/pacman-extra.conf")
parser.add_argument("--no-multilib", help="do not add multilib repository", action="store_true")
parser.add_argument("--packager", help="packager name and email", required=True)
parser.add_argument("--repository", help="repository name", default="aur-clone")
parser.set_defaults(handler=handlers.Setup, lock=None, no_report=True, unsafe=True)
return parser
def _set_sign_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for sign subcommand
:param root: subparsers for the commands
:return: created argument parser
"""
parser = root.add_parser("sign", help="sign packages", description="(re-)sign packages and repository database",
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument("package", help="sign only specified packages", nargs="*")
parser.set_defaults(handler=handlers.Sign)
return parser
def _set_status_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for status subcommand
:param root: subparsers for the commands
:return: created argument parser
"""
parser = root.add_parser("status", help="get package status", description="request status of the package",
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument("--ahriman", help="get service status itself", action="store_true")
parser.add_argument("package", help="filter status by package base", nargs="*")
parser.set_defaults(handler=handlers.Status, lock=None, no_report=True, unsafe=True)
return parser
def _set_status_update_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for status update subcommand
:param root: subparsers for the commands
:return: created argument parser
"""
parser = root.add_parser("status-update", help="update package status", description="request status of the package",
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument(
"package",
help="set status for specified packages. If no packages supplied, service status will be updated",
nargs="*")
parser.add_argument("--status", help="new status", choices=[value.value for value in BuildStatusEnum],
default="success")
parser.set_defaults(handler=handlers.StatusUpdate, lock=None, no_report=True, unsafe=True)
return parser
def _set_sync_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for sync subcommand
:param root: subparsers for the commands
:return: created argument parser
"""
parser = root.add_parser("sync", help="sync repository", description="sync packages to remote server",
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument("target", help="target to sync", nargs="*")
parser.set_defaults(handler=handlers.Sync)
return parser
def _set_update_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for update subcommand
:param root: subparsers for the commands
:return: created argument parser
"""
parser = root.add_parser("update", help="update packages", description="run updates",
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument("package", help="filter check by package base", nargs="*")
parser.add_argument("--dry-run", help="just perform check for updates, same as check command", action="store_true")
parser.add_argument("--no-aur", help="do not check for AUR updates. Implies --no-vcs", action="store_true")
parser.add_argument("--no-manual", help="do not include manual updates", action="store_true")
parser.add_argument("--no-vcs", help="do not check VCS packages", action="store_true")
parser.set_defaults(handler=handlers.Update)
return parser
def _set_web_parser(root: SubParserAction) -> argparse.ArgumentParser:
"""
add parser for web subcommand
:param root: subparsers for the commands
:return: created argument parser
"""
parser = root.add_parser("web", help="start web server", description="start web server",
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.set_defaults(handler=handlers.Web, lock=None, no_report=True)
return parser return parser

View File

@ -35,21 +35,21 @@ class Application:
""" """
base application class base application class
:ivar architecture: repository architecture :ivar architecture: repository architecture
:ivar config: configuration instance :ivar configuration: configuration instance
:ivar logger: application logger :ivar logger: application logger
:ivar repository: repository instance :ivar repository: repository instance
""" """
def __init__(self, architecture: str, config: Configuration) -> None: def __init__(self, architecture: str, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
self.logger = logging.getLogger("root") self.logger = logging.getLogger("root")
self.config = config self.configuration = configuration
self.architecture = architecture self.architecture = architecture
self.repository = Repository(architecture, config) self.repository = Repository(architecture, configuration)
def _known_packages(self) -> Set[str]: def _known_packages(self) -> Set[str]:
""" """
@ -106,7 +106,7 @@ class Application:
add_archive(full_path) add_archive(full_path)
def add_manual(src: str) -> Path: def add_manual(src: str) -> Path:
package = Package.load(src, self.repository.pacman, self.config.get("alpm", "aur_url")) package = Package.load(src, self.repository.pacman, self.configuration.get("alpm", "aur_url"))
path = self.repository.paths.manual / package.base path = self.repository.paths.manual / package.base
Task.fetch(path, package.git_url) Task.fetch(path, package.git_url)
return path return path

View File

@ -28,6 +28,7 @@ from ahriman.application.handlers.report import Report
from ahriman.application.handlers.setup import Setup from ahriman.application.handlers.setup import Setup
from ahriman.application.handlers.sign import Sign from ahriman.application.handlers.sign import Sign
from ahriman.application.handlers.status import Status from ahriman.application.handlers.status import Status
from ahriman.application.handlers.status_update import StatusUpdate
from ahriman.application.handlers.sync import Sync from ahriman.application.handlers.sync import Sync
from ahriman.application.handlers.update import Update from ahriman.application.handlers.update import Update
from ahriman.application.handlers.web import Web from ahriman.application.handlers.web import Web

View File

@ -32,11 +32,17 @@ class Add(Handler):
""" """
@classmethod @classmethod
def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, config: Configuration) -> None: def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, configuration: Configuration) -> None:
""" """
callback for command line callback for command line
:param args: command line args :param args: command line args
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
Application(architecture, config).add(args.package, args.without_dependencies) application = Application(architecture, configuration)
application.add(args.package, args.without_dependencies)
if not args.now:
return
packages = application.get_updates(args.package, True, False, True, application.logger.info)
application.update(packages)

View File

@ -32,12 +32,12 @@ class Clean(Handler):
""" """
@classmethod @classmethod
def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, config: Configuration) -> None: def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, configuration: Configuration) -> None:
""" """
callback for command line callback for command line
:param args: command line args :param args: command line args
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
Application(architecture, config).clean(args.no_build, args.no_cache, args.no_chroot, Application(architecture, configuration).clean(args.no_build, args.no_cache, args.no_chroot,
args.no_manual, args.no_packages) args.no_manual, args.no_packages)

View File

@ -27,20 +27,22 @@ from ahriman.core.configuration import Configuration
class Dump(Handler): class Dump(Handler):
""" """
dump config handler dump configuration handler
""" """
_print = print
@classmethod @classmethod
def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, config: Configuration) -> None: def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, configuration: Configuration) -> None:
""" """
callback for command line callback for command line
:param args: command line args :param args: command line args
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
config_dump = config.dump(architecture) dump = configuration.dump()
for section, values in sorted(config_dump.items()): for section, values in sorted(dump.items()):
print(f"[{section}]") Dump._print(f"[{section}]")
for key, value in sorted(values.items()): for key, value in sorted(values.items()):
print(f"{key} = {value}") Dump._print(f"{key} = {value}")
print() Dump._print()

View File

@ -35,17 +35,17 @@ class Handler:
""" """
@classmethod @classmethod
def _call(cls: Type[Handler], args: argparse.Namespace, architecture: str, config: Configuration) -> bool: def _call(cls: Type[Handler], args: argparse.Namespace, architecture: str) -> bool:
""" """
additional function to wrap all calls for multiprocessing library additional function to wrap all calls for multiprocessing library
:param args: command line args :param args: command line args
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance
:return: True on success, False otherwise :return: True on success, False otherwise
""" """
try: try:
with Lock(args, architecture, config): configuration = Configuration.from_path(args.configuration, architecture, not args.no_log)
cls.run(args, architecture, config) with Lock(args, architecture, configuration):
cls.run(args, architecture, configuration)
return True return True
except Exception: except Exception:
logging.getLogger("root").exception("process exception") logging.getLogger("root").exception("process exception")
@ -58,18 +58,17 @@ class Handler:
:param args: command line args :param args: command line args
:return: 0 on success, 1 otherwise :return: 0 on success, 1 otherwise
""" """
configuration = Configuration.from_path(args.config, not args.no_log)
with Pool(len(args.architecture)) as pool: with Pool(len(args.architecture)) as pool:
result = pool.starmap( result = pool.starmap(
cls._call, [(args, architecture, configuration) for architecture in args.architecture]) cls._call, [(args, architecture) for architecture in set(args.architecture)])
return 0 if all(result) else 1 return 0 if all(result) else 1
@classmethod @classmethod
def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, config: Configuration) -> None: def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, configuration: Configuration) -> None:
""" """
callback for command line callback for command line
:param args: command line args :param args: command line args
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
raise NotImplementedError raise NotImplementedError

View File

@ -32,13 +32,13 @@ class Rebuild(Handler):
""" """
@classmethod @classmethod
def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, config: Configuration) -> None: def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, configuration: Configuration) -> None:
""" """
callback for command line callback for command line
:param args: command line args :param args: command line args
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
application = Application(architecture, config) application = Application(architecture, configuration)
packages = application.repository.packages() packages = application.repository.packages()
application.update(packages) application.update(packages)

View File

@ -32,11 +32,11 @@ class Remove(Handler):
""" """
@classmethod @classmethod
def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, config: Configuration) -> None: def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, configuration: Configuration) -> None:
""" """
callback for command line callback for command line
:param args: command line args :param args: command line args
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
Application(architecture, config).remove(args.package) Application(architecture, configuration).remove(args.package)

View File

@ -32,11 +32,11 @@ class Report(Handler):
""" """
@classmethod @classmethod
def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, config: Configuration) -> None: def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, configuration: Configuration) -> None:
""" """
callback for command line callback for command line
:param args: command line args :param args: command line args
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
Application(architecture, config).report(args.target) Application(architecture, configuration).report(args.target)

View File

@ -44,19 +44,19 @@ class Setup(Handler):
SUDOERS_PATH = Path("/etc/sudoers.d/ahriman") SUDOERS_PATH = Path("/etc/sudoers.d/ahriman")
@classmethod @classmethod
def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, config: Configuration) -> None: def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, configuration: Configuration) -> None:
""" """
callback for command line callback for command line
:param args: command line args :param args: command line args
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
application = Application(architecture, config) application = Application(architecture, configuration)
Setup.create_makepkg_configuration(args.packager, application.repository.paths) Setup.create_makepkg_configuration(args.packager, application.repository.paths)
Setup.create_executable(args.build_command, architecture) Setup.create_executable(args.build_command, architecture)
Setup.create_devtools_configuration(args.build_command, architecture, Path(args.from_config), args.no_multilib, Setup.create_devtools_configuration(args.build_command, architecture, Path(args.from_configuration),
args.repository, application.repository.paths) args.no_multilib, args.repository, application.repository.paths)
Setup.create_ahriman_configuration(args.build_command, architecture, args.repository, config.include) Setup.create_ahriman_configuration(args.build_command, architecture, args.repository, configuration.include)
Setup.create_sudo_configuration(args.build_command, architecture) Setup.create_sudo_configuration(args.build_command, architecture)
@staticmethod @staticmethod
@ -78,17 +78,17 @@ class Setup(Handler):
:param repository: repository name :param repository: repository name
:param include_path: path to directory with configuration includes :param include_path: path to directory with configuration includes
""" """
config = configparser.ConfigParser() configuration = configparser.ConfigParser()
config.add_section("build") configuration.add_section("build")
config.set("build", "build_command", str(Setup.build_command(prefix, architecture))) configuration.set("build", "build_command", str(Setup.build_command(prefix, architecture)))
config.add_section("repository") configuration.add_section("repository")
config.set("repository", "name", repository) configuration.set("repository", "name", repository)
target = include_path / "build-overrides.ini" target = include_path / "build-overrides.ini"
with target.open("w") as ahriman_config: with target.open("w") as ahriman_configuration:
config.write(ahriman_config) configuration.write(ahriman_configuration)
@staticmethod @staticmethod
def create_devtools_configuration(prefix: str, architecture: str, source: Path, def create_devtools_configuration(prefix: str, architecture: str, source: Path,
@ -102,31 +102,31 @@ class Setup(Handler):
:param repository: repository name :param repository: repository name
:param paths: repository paths instance :param paths: repository paths instance
""" """
config = configparser.ConfigParser() configuration = configparser.ConfigParser()
# preserve case # preserve case
# stupid mypy thinks that it is impossible # stupid mypy thinks that it is impossible
config.optionxform = lambda key: key # type: ignore configuration.optionxform = lambda key: key # type: ignore
# load default configuration first # load default configuration first
# we cannot use Include here because it will be copied to new chroot, thus no includes there # we cannot use Include here because it will be copied to new chroot, thus no includes there
config.read(source) configuration.read(source)
# set our architecture now # set our architecture now
config.set("options", "Architecture", architecture) configuration.set("options", "Architecture", architecture)
# add multilib # add multilib
if not no_multilib: if not no_multilib:
config.add_section("multilib") configuration.add_section("multilib")
config.set("multilib", "Include", str(Setup.MIRRORLIST_PATH)) configuration.set("multilib", "Include", str(Setup.MIRRORLIST_PATH))
# add repository itself # add repository itself
config.add_section(repository) configuration.add_section(repository)
config.set(repository, "SigLevel", "Optional TrustAll") # we don't care configuration.set(repository, "SigLevel", "Optional TrustAll") # we don't care
config.set(repository, "Server", f"file://{paths.repository}") configuration.set(repository, "Server", f"file://{paths.repository}")
target = source.parent / f"pacman-{prefix}.conf" target = source.parent / f"pacman-{prefix}-{architecture}.conf"
with target.open("w") as devtools_config: with target.open("w") as devtools_configuration:
config.write(devtools_config) configuration.write(devtools_configuration)
@staticmethod @staticmethod
def create_makepkg_configuration(packager: str, paths: RepositoryPaths) -> None: def create_makepkg_configuration(packager: str, paths: RepositoryPaths) -> None:

View File

@ -32,11 +32,11 @@ class Sign(Handler):
""" """
@classmethod @classmethod
def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, config: Configuration) -> None: def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, configuration: Configuration) -> None:
""" """
callback for command line callback for command line
:param args: command line args :param args: command line args
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
Application(architecture, config).sign(args.package) Application(architecture, configuration).sign(args.package)

View File

@ -34,14 +34,14 @@ class Status(Handler):
""" """
@classmethod @classmethod
def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, config: Configuration) -> None: def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, configuration: Configuration) -> None:
""" """
callback for command line callback for command line
:param args: command line args :param args: command line args
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
application = Application(architecture, config) application = Application(architecture, configuration)
if args.ahriman: if args.ahriman:
ahriman = application.repository.reporter.get_self() ahriman = application.repository.reporter.get_self()
print(ahriman.pretty_print()) print(ahriman.pretty_print())

View File

@ -0,0 +1,51 @@
#
# Copyright (c) 2021 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
from typing import Type
from ahriman.application.application import Application
from ahriman.application.handlers.handler import Handler
from ahriman.core.configuration import Configuration
from ahriman.models.build_status import BuildStatusEnum
class StatusUpdate(Handler):
"""
status update handler
"""
@classmethod
def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, configuration: Configuration) -> None:
"""
callback for command line
:param args: command line args
:param architecture: repository architecture
:param configuration: configuration instance
"""
client = Application(architecture, configuration).repository.reporter
status = BuildStatusEnum(args.status)
if args.package:
# update packages statuses
for package in args.package:
client.update(package, status)
else:
# update service status
client.update_self(status)

View File

@ -32,11 +32,11 @@ class Sync(Handler):
""" """
@classmethod @classmethod
def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, config: Configuration) -> None: def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, configuration: Configuration) -> None:
""" """
callback for command line callback for command line
:param args: command line args :param args: command line args
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
Application(architecture, config).sync(args.target) Application(architecture, configuration).sync(args.target)

View File

@ -32,18 +32,18 @@ class Update(Handler):
""" """
@classmethod @classmethod
def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, config: Configuration) -> None: def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, configuration: Configuration) -> None:
""" """
callback for command line callback for command line
:param args: command line args :param args: command line args
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
# typing workaround # typing workaround
def log_fn(line: str) -> None: def log_fn(line: str) -> None:
return print(line) if args.dry_run else application.logger.info(line) return print(line) if args.dry_run else application.logger.info(line)
application = Application(architecture, config) application = Application(architecture, configuration)
packages = application.get_updates(args.package, args.no_aur, args.no_manual, args.no_vcs, log_fn) packages = application.get_updates(args.package, args.no_aur, args.no_manual, args.no_vcs, log_fn)
if args.dry_run: if args.dry_run:
return return

View File

@ -31,13 +31,13 @@ class Web(Handler):
""" """
@classmethod @classmethod
def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, config: Configuration) -> None: def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, configuration: Configuration) -> None:
""" """
callback for command line callback for command line
:param args: command line args :param args: command line args
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
from ahriman.web.web import run_server, setup_service from ahriman.web.web import run_server, setup_service
application = setup_service(architecture, config) application = setup_service(architecture, configuration)
run_server(application) run_server(application)

View File

@ -42,19 +42,19 @@ class Lock:
:ivar unsafe: skip user check :ivar unsafe: skip user check
""" """
def __init__(self, args: argparse.Namespace, architecture: str, config: Configuration) -> None: def __init__(self, args: argparse.Namespace, architecture: str, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param args: command line args :param args: command line args
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
self.path = Path(f"{args.lock}_{architecture}") if args.lock is not None else None self.path = Path(f"{args.lock}_{architecture}") if args.lock is not None else None
self.force = args.force self.force = args.force
self.unsafe = args.unsafe self.unsafe = args.unsafe
self.root = Path(config.get("repository", "root")) self.root = Path(configuration.get("repository", "root"))
self.reporter = Client() if args.no_report else Client.load(architecture, config) self.reporter = Client() if args.no_report else Client.load(configuration)
def __enter__(self) -> Lock: def __enter__(self) -> Lock:
""" """
@ -80,7 +80,7 @@ class Lock:
:param exc_tb: exception traceback if any :param exc_tb: exception traceback if any
:return: always False (do not suppress any exception) :return: always False (do not suppress any exception)
""" """
self.remove() self.clear()
status = BuildStatusEnum.Success if exc_val is None else BuildStatusEnum.Failed status = BuildStatusEnum.Success if exc_val is None else BuildStatusEnum.Failed
self.reporter.update_self(status) self.reporter.update_self(status)
return False return False
@ -96,6 +96,14 @@ class Lock:
if current_uid != root_uid: if current_uid != root_uid:
raise UnsafeRun(current_uid, root_uid) raise UnsafeRun(current_uid, root_uid)
def clear(self) -> None:
"""
remove lock file
"""
if self.path is None:
return
self.path.unlink(missing_ok=True)
def create(self) -> None: def create(self) -> None:
""" """
create lock file create lock file
@ -106,11 +114,3 @@ class Lock:
self.path.touch(exist_ok=self.force) self.path.touch(exist_ok=self.force)
except FileExistsError: except FileExistsError:
raise DuplicateRun() raise DuplicateRun()
def remove(self) -> None:
"""
remove lock file
"""
if self.path is None:
return
self.path.unlink(missing_ok=True)

View File

@ -29,15 +29,15 @@ class Pacman:
:ivar handle: pyalpm root `Handle` :ivar handle: pyalpm root `Handle`
""" """
def __init__(self, config: Configuration) -> None: def __init__(self, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param config: configuration instance :param configuration: configuration instance
""" """
root = config.get("alpm", "root") root = configuration.get("alpm", "root")
pacman_root = config.getpath("alpm", "database") pacman_root = configuration.getpath("alpm", "database")
self.handle = Handle(root, str(pacman_root)) self.handle = Handle(root, str(pacman_root))
for repository in config.getlist("alpm", "repositories"): for repository in configuration.getlist("alpm", "repositories"):
self.handle.register_syncdb(repository, 0) # 0 is pgp_level self.handle.register_syncdb(repository, 0) # 0 is pgp_level
def all_packages(self) -> List[str]: def all_packages(self) -> List[str]:

View File

@ -41,12 +41,11 @@ class Task:
_check_output = check_output _check_output = check_output
def __init__(self, package: Package, architecture: str, config: Configuration, paths: RepositoryPaths) -> None: def __init__(self, package: Package, configuration: Configuration, paths: RepositoryPaths) -> None:
""" """
default constructor default constructor
:param package: package definitions :param package: package definitions
:param architecture: repository architecture :param configuration: configuration instance
:param config: configuration instance
:param paths: repository paths instance :param paths: repository paths instance
""" """
self.logger = logging.getLogger("builder") self.logger = logging.getLogger("builder")
@ -54,11 +53,10 @@ class Task:
self.package = package self.package = package
self.paths = paths self.paths = paths
section = config.get_section_name("build", architecture) self.archbuild_flags = configuration.getlist("build", "archbuild_flags")
self.archbuild_flags = config.getlist(section, "archbuild_flags") self.build_command = configuration.get("build", "build_command")
self.build_command = config.get(section, "build_command") self.makepkg_flags = configuration.getlist("build", "makepkg_flags")
self.makepkg_flags = config.getlist(section, "makepkg_flags") self.makechrootpkg_flags = configuration.getlist("build", "makechrootpkg_flags")
self.makechrootpkg_flags = config.getlist(section, "makechrootpkg_flags")
@property @property
def cache_path(self) -> Path: def cache_path(self) -> Path:
@ -97,14 +95,14 @@ class Task:
run package build run package build
:return: paths of produced packages :return: paths of produced packages
""" """
cmd = [self.build_command, "-r", str(self.paths.chroot)] command = [self.build_command, "-r", str(self.paths.chroot)]
cmd.extend(self.archbuild_flags) command.extend(self.archbuild_flags)
cmd.extend(["--"] + self.makechrootpkg_flags) command.extend(["--"] + self.makechrootpkg_flags)
cmd.extend(["--"] + self.makepkg_flags) command.extend(["--"] + self.makepkg_flags)
self.logger.info(f"using {cmd} for {self.package.base}") self.logger.info(f"using {command} for {self.package.base}")
Task._check_output( Task._check_output(
*cmd, *command,
exception=BuildFailed(self.package.base), exception=BuildFailed(self.package.base),
cwd=self.git_path, cwd=self.git_path,
logger=self.build_logger) logger=self.build_logger)
@ -125,4 +123,4 @@ class Task:
if self.cache_path.is_dir(): if self.cache_path.is_dir():
# no need to clone whole repository, just copy from cache first # no need to clone whole repository, just copy from cache first
shutil.copytree(self.cache_path, git_path) shutil.copytree(self.cache_path, git_path)
return Task.fetch(git_path, self.package.git_url) return self.fetch(git_path, self.package.git_url)

View File

@ -34,13 +34,11 @@ class Configuration(configparser.RawConfigParser):
:cvar ARCHITECTURE_SPECIFIC_SECTIONS: known sections which can be architecture specific (required by dump) :cvar ARCHITECTURE_SPECIFIC_SECTIONS: known sections which can be architecture specific (required by dump)
:cvar DEFAULT_LOG_FORMAT: default log format (in case of fallback) :cvar DEFAULT_LOG_FORMAT: default log format (in case of fallback)
:cvar DEFAULT_LOG_LEVEL: default log level (in case of fallback) :cvar DEFAULT_LOG_LEVEL: default log level (in case of fallback)
:cvar STATIC_SECTIONS: known sections which are not architecture specific (required by dump)
""" """
DEFAULT_LOG_FORMAT = "[%(levelname)s %(asctime)s] [%(filename)s:%(lineno)d] [%(funcName)s]: %(message)s" DEFAULT_LOG_FORMAT = "[%(levelname)s %(asctime)s] [%(filename)s:%(lineno)d] [%(funcName)s]: %(message)s"
DEFAULT_LOG_LEVEL = logging.DEBUG DEFAULT_LOG_LEVEL = logging.DEBUG
STATIC_SECTIONS = ["alpm", "report", "repository", "settings", "upload"]
ARCHITECTURE_SPECIFIC_SECTIONS = ["build", "html", "rsync", "s3", "sign", "web"] ARCHITECTURE_SPECIFIC_SECTIONS = ["build", "html", "rsync", "s3", "sign", "web"]
def __init__(self) -> None: def __init__(self) -> None:
@ -57,37 +55,46 @@ class Configuration(configparser.RawConfigParser):
""" """
return self.getpath("settings", "include") return self.getpath("settings", "include")
@property
def logging_path(self) -> Path:
"""
:return: path to logging configuration
"""
return self.getpath("settings", "logging")
@classmethod @classmethod
def from_path(cls: Type[Configuration], path: Path, logfile: bool) -> Configuration: def from_path(cls: Type[Configuration], path: Path, architecture: str, logfile: bool) -> Configuration:
""" """
constructor with full object initialization constructor with full object initialization
:param path: path to root configuration file :param path: path to root configuration file
:param architecture: repository architecture
:param logfile: use log file to output messages :param logfile: use log file to output messages
:return: configuration instance :return: configuration instance
""" """
config = cls() config = cls()
config.load(path) config.load(path, architecture)
config.load_logging(logfile) config.load_logging(logfile)
return config return config
def dump(self, architecture: str) -> Dict[str, Dict[str, str]]: @staticmethod
def section_name(section: str, architecture: str) -> str:
"""
generate section name for architecture specific sections
:param section: section name
:param architecture: repository architecture
:return: correct section name for repository specific section
"""
return f"{section}:{architecture}"
def dump(self) -> Dict[str, Dict[str, str]]:
""" """
dump configuration to dictionary dump configuration to dictionary
:param architecture: repository architecture
:return: configuration dump for specific architecture :return: configuration dump for specific architecture
""" """
result: Dict[str, Dict[str, str]] = {} return {
for section in Configuration.STATIC_SECTIONS: section: dict(self[section])
if not self.has_section(section): for section in self.sections()
continue }
result[section] = dict(self[section])
for group in Configuration.ARCHITECTURE_SPECIFIC_SECTIONS:
section = self.get_section_name(group, architecture)
if not self.has_section(section):
continue
result[section] = dict(self[section])
return result
def getlist(self, section: str, key: str) -> List[str]: def getlist(self, section: str, key: str) -> List[str]:
""" """
@ -113,24 +120,16 @@ class Configuration(configparser.RawConfigParser):
return value return value
return self.path.parent / value return self.path.parent / value
def get_section_name(self, prefix: str, suffix: str) -> str: def load(self, path: Path, architecture: str) -> None:
"""
check if there is `prefix`_`suffix` section and return it on success. Return `prefix` otherwise
:param prefix: section name prefix
:param suffix: section name suffix (e.g. architecture name)
:return: found section name
"""
probe = f"{prefix}_{suffix}"
return probe if self.has_section(probe) else prefix
def load(self, path: Path) -> None:
""" """
fully load configuration fully load configuration
:param path: path to root configuration file :param path: path to root configuration file
:param architecture: repository architecture
""" """
self.path = path self.path = path
self.read(self.path) self.read(self.path)
self.load_includes() self.load_includes()
self.merge_sections(architecture)
def load_includes(self) -> None: def load_includes(self) -> None:
""" """
@ -138,6 +137,8 @@ class Configuration(configparser.RawConfigParser):
""" """
try: try:
for path in sorted(self.include.glob("*.ini")): for path in sorted(self.include.glob("*.ini")):
if path == self.logging_path:
continue # we don't want to load logging explicitly
self.read(path) self.read(path)
except (FileNotFoundError, configparser.NoOptionError): except (FileNotFoundError, configparser.NoOptionError):
pass pass
@ -149,17 +150,39 @@ class Configuration(configparser.RawConfigParser):
""" """
def file_logger() -> None: def file_logger() -> None:
try: try:
config_path = self.getpath("settings", "logging") path = self.logging_path
fileConfig(config_path) fileConfig(path)
except (FileNotFoundError, PermissionError): except (FileNotFoundError, PermissionError):
console_logger() console_logger()
logging.exception("could not create logfile, fallback to stderr") logging.exception("could not create logfile, fallback to stderr")
def console_logger() -> None: def console_logger() -> None:
logging.basicConfig(filename=None, format=Configuration.DEFAULT_LOG_FORMAT, logging.basicConfig(filename=None, format=self.DEFAULT_LOG_FORMAT,
level=Configuration.DEFAULT_LOG_LEVEL) level=self.DEFAULT_LOG_LEVEL)
if logfile: if logfile:
file_logger() file_logger()
else: else:
console_logger() console_logger()
def merge_sections(self, architecture: str) -> None:
"""
merge architecture specific sections into main configuration
:param architecture: repository architecture
"""
for section in self.ARCHITECTURE_SPECIFIC_SECTIONS:
if not self.has_section(section):
self.add_section(section) # add section if not exists
# get overrides
specific = self.section_name(section, architecture)
if self.has_section(specific):
# if there is no such section it means that there is no overrides for this arch
# but we anyway will have to delete sections for others archs
for key, value in self[specific].items():
self.set(section, key, value)
# remove any arch specific section
for foreign in self.sections():
# we would like to use lambda filter here, but pylint is too dumb
if not foreign.startswith(f"{section}:"):
continue
self.remove_section(foreign)

View File

@ -23,7 +23,8 @@ from typing import Callable, Dict, Iterable
from ahriman.core.configuration import Configuration from ahriman.core.configuration import Configuration
from ahriman.core.report.report import Report from ahriman.core.report.report import Report
from ahriman.core.util import pretty_size, pretty_datetime from ahriman.core.sign.gpg import GPG
from ahriman.core.util import pretty_datetime, pretty_size
from ahriman.models.package import Package from ahriman.models.package import Package
from ahriman.models.sign_settings import SignSettings from ahriman.models.sign_settings import SignSettings
@ -38,38 +39,46 @@ class HTML(Report):
link_path - prefix fo packages to download, string, required link_path - prefix fo packages to download, string, required
has_package_signed - True in case if package sign enabled, False otherwise, required has_package_signed - True in case if package sign enabled, False otherwise, required
has_repo_signed - True in case if repository database sign enabled, False otherwise, required has_repo_signed - True in case if repository database sign enabled, False otherwise, required
packages - sorted list of packages properties: archive_size, build_date, filename, installed_size, name, version. Required packages - sorted list of packages properties, required
* architecture, string
* archive_size, pretty printed size, string
* build_date, pretty printed datetime, string
* description, string
* filename, string,
* groups, sorted list of strings
* installed_size, pretty printed datetime, string
* licenses, sorted list of strings
* name, string
* url, string
* version, string
pgp_key - default PGP key ID, string, optional pgp_key - default PGP key ID, string, optional
repository - repository name, string, required repository - repository name, string, required
:ivar homepage: homepage link if any (for footer) :ivar homepage: homepage link if any (for footer)
:ivar link_path: prefix fo packages to download :ivar link_path: prefix fo packages to download
:ivar name: repository name :ivar name: repository name
:ivar pgp_key: default PGP key :ivar default_pgp_key: default PGP key
:ivar report_path: output path to html report :ivar report_path: output path to html report
:ivar sign_targets: targets to sign enabled in configuration :ivar sign_targets: targets to sign enabled in configuration
:ivar tempate_path: path to directory with jinja templates :ivar template_path: path to directory with jinja templates
""" """
def __init__(self, architecture: str, config: Configuration) -> None: def __init__(self, architecture: str, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
Report.__init__(self, architecture, config) Report.__init__(self, architecture, configuration)
section = config.get_section_name("html", architecture) self.report_path = configuration.getpath("html", "path")
self.report_path = config.getpath(section, "path") self.link_path = configuration.get("html", "link_path")
self.link_path = config.get(section, "link_path") self.template_path = configuration.getpath("html", "template_path")
self.template_path = config.getpath(section, "template_path")
# base template vars # base template vars
self.homepage = config.get(section, "homepage", fallback=None) self.homepage = configuration.get("html", "homepage", fallback=None)
self.name = config.get("repository", "name") self.name = configuration.get("repository", "name")
sign_section = config.get_section_name("sign", architecture) self.sign_targets, self.default_pgp_key = GPG.sign_options(configuration)
self.sign_targets = [SignSettings.from_option(opt) for opt in config.getlist(sign_section, "target")]
self.pgp_key = config.get(sign_section, "key") if self.sign_targets else None
def generate(self, packages: Iterable[Package]) -> None: def generate(self, packages: Iterable[Package]) -> None:
""" """
@ -83,11 +92,16 @@ class HTML(Report):
content = [ content = [
{ {
"architecture": properties.architecture or "",
"archive_size": pretty_size(properties.archive_size), "archive_size": pretty_size(properties.archive_size),
"build_date": pretty_datetime(properties.build_date), "build_date": pretty_datetime(properties.build_date),
"description": properties.description or "",
"filename": properties.filename, "filename": properties.filename,
"groups": properties.groups,
"installed_size": pretty_size(properties.installed_size), "installed_size": pretty_size(properties.installed_size),
"licenses": properties.licenses,
"name": package, "name": package,
"url": properties.url or "",
"version": base.version "version": base.version
} for base in packages for package, properties in base.packages.items() } for base in packages for package, properties in base.packages.items()
] ]
@ -99,7 +113,7 @@ class HTML(Report):
has_package_signed=SignSettings.SignPackages in self.sign_targets, has_package_signed=SignSettings.SignPackages in self.sign_targets,
has_repo_signed=SignSettings.SignRepository in self.sign_targets, has_repo_signed=SignSettings.SignRepository in self.sign_targets,
packages=sorted(content, key=comparator), packages=sorted(content, key=comparator),
pgp_key=self.pgp_key, pgp_key=self.default_pgp_key,
repository=self.name) repository=self.name)
self.report_path.write_text(html) self.report_path.write_text(html)

View File

@ -17,9 +17,11 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
# #
from __future__ import annotations
import logging import logging
from typing import Iterable from typing import Iterable, Type
from ahriman.core.configuration import Configuration from ahriman.core.configuration import Configuration
from ahriman.core.exceptions import ReportFailed from ahriman.core.exceptions import ReportFailed
@ -31,44 +33,48 @@ class Report:
""" """
base report generator base report generator
:ivar architecture: repository architecture :ivar architecture: repository architecture
:ivar config: configuration instance :ivar configuration: configuration instance
:ivar logger: class logger :ivar logger: class logger
""" """
def __init__(self, architecture: str, config: Configuration) -> None: def __init__(self, architecture: str, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
self.logger = logging.getLogger("builder") self.logger = logging.getLogger("builder")
self.architecture = architecture self.architecture = architecture
self.config = config self.configuration = configuration
@staticmethod @classmethod
def run(architecture: str, config: Configuration, target: str, packages: Iterable[Package]) -> None: def load(cls: Type[Report], architecture: str, configuration: Configuration, target: str) -> Report:
""" """
run report generation load client from settings
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
:param target: target to generate report (e.g. html) :param target: target to generate report (e.g. html)
:param packages: list of packages to generate report :return: client according to current settings
""" """
provider = ReportSettings.from_option(target) provider = ReportSettings.from_option(target)
if provider == ReportSettings.HTML: if provider == ReportSettings.HTML:
from ahriman.core.report.html import HTML from ahriman.core.report.html import HTML
report: Report = HTML(architecture, config) return HTML(architecture, configuration)
else: return cls(architecture, configuration) # should never happen
report = Report(architecture, config)
try:
report.generate(packages)
except Exception:
report.logger.exception(f"report generation failed for target {provider.name}")
raise ReportFailed()
def generate(self, packages: Iterable[Package]) -> None: def generate(self, packages: Iterable[Package]) -> None:
""" """
generate report for the specified packages generate report for the specified packages
:param packages: list of packages to generate report :param packages: list of packages to generate report
""" """
def run(self, packages: Iterable[Package]) -> None:
"""
run report generation
:param packages: list of packages to generate report
"""
try:
self.generate(packages)
except Exception:
self.logger.exception("report generation failed")
raise ReportFailed()

View File

@ -25,7 +25,7 @@ from typing import Dict, Iterable, List, Optional
from ahriman.core.build_tools.task import Task from ahriman.core.build_tools.task import Task
from ahriman.core.report.report import Report from ahriman.core.report.report import Report
from ahriman.core.repository.cleaner import Cleaner from ahriman.core.repository.cleaner import Cleaner
from ahriman.core.upload.uploader import Uploader from ahriman.core.upload.upload import Upload
from ahriman.models.package import Package from ahriman.models.package import Package
@ -49,7 +49,7 @@ class Executor(Cleaner):
""" """
def build_single(package: Package) -> None: def build_single(package: Package) -> None:
self.reporter.set_building(package.base) self.reporter.set_building(package.base)
task = Task(package, self.architecture, self.config, self.paths) task = Task(package, self.configuration, self.paths)
task.init() task.init()
built = task.build() built = task.build()
for src in built: for src in built:
@ -106,9 +106,10 @@ class Executor(Cleaner):
:param targets: list of targets to generate reports. Configuration option will be used if it is not set :param targets: list of targets to generate reports. Configuration option will be used if it is not set
""" """
if targets is None: if targets is None:
targets = self.config.getlist("report", "target") targets = self.configuration.getlist("report", "target")
for target in targets: for target in targets:
Report.run(self.architecture, self.config, target, self.packages()) runner = Report.load(self.architecture, self.configuration, target)
runner.run(self.packages())
def process_sync(self, targets: Optional[Iterable[str]]) -> None: def process_sync(self, targets: Optional[Iterable[str]]) -> None:
""" """
@ -116,9 +117,10 @@ class Executor(Cleaner):
:param targets: list of targets to sync. Configuration option will be used if it is not set :param targets: list of targets to sync. Configuration option will be used if it is not set
""" """
if targets is None: if targets is None:
targets = self.config.getlist("upload", "target") targets = self.configuration.getlist("upload", "target")
for target in targets: for target in targets:
Uploader.run(self.architecture, self.config, target, self.paths.repository) runner = Upload.load(self.architecture, self.configuration, target)
runner.run(self.paths.repository)
def process_update(self, packages: Iterable[Path]) -> Path: def process_update(self, packages: Iterable[Path]) -> Path:
""" """

View File

@ -32,7 +32,8 @@ class Properties:
repository internal objects holder repository internal objects holder
:ivar architecture: repository architecture :ivar architecture: repository architecture
:ivar aur_url: base AUR url :ivar aur_url: base AUR url
:ivar config: configuration instance :ivar configuration: configuration instance
:ivar ignore_list: package bases which will be ignored during auto updates
:ivar logger: class logger :ivar logger: class logger
:ivar name: repository name :ivar name: repository name
:ivar pacman: alpm wrapper instance :ivar pacman: alpm wrapper instance
@ -42,18 +43,19 @@ class Properties:
:ivar sign: GPG wrapper instance :ivar sign: GPG wrapper instance
""" """
def __init__(self, architecture: str, config: Configuration) -> None: def __init__(self, architecture: str, configuration: Configuration) -> None:
self.logger = logging.getLogger("builder") self.logger = logging.getLogger("builder")
self.architecture = architecture self.architecture = architecture
self.config = config self.configuration = configuration
self.aur_url = config.get("alpm", "aur_url") self.aur_url = configuration.get("alpm", "aur_url")
self.name = config.get("repository", "name") self.name = configuration.get("repository", "name")
self.paths = RepositoryPaths(config.getpath("repository", "root"), architecture) self.paths = RepositoryPaths(configuration.getpath("repository", "root"), architecture)
self.paths.create_tree() self.paths.create_tree()
self.pacman = Pacman(config) self.ignore_list = configuration.getlist("build", "ignore_packages")
self.sign = GPG(architecture, config) self.pacman = Pacman(configuration)
self.sign = GPG(architecture, configuration)
self.repo = Repo(self.name, self.paths, self.sign.repository_sign_args) self.repo = Repo(self.name, self.paths, self.sign.repository_sign_args)
self.reporter = Client.load(architecture, config) self.reporter = Client.load(configuration)

View File

@ -44,11 +44,8 @@ class UpdateHandler(Cleaner):
""" """
result: List[Package] = [] result: List[Package] = []
build_section = self.config.get_section_name("build", self.architecture)
ignore_list = self.config.getlist(build_section, "ignore_packages")
for local in self.packages(): for local in self.packages():
if local.base in ignore_list: if local.base in self.ignore_list:
continue continue
if local.is_vcs and no_vcs: if local.is_vcs and no_vcs:
continue continue

View File

@ -20,7 +20,7 @@
import logging import logging
from pathlib import Path from pathlib import Path
from typing import List from typing import List, Optional, Set, Tuple
from ahriman.core.configuration import Configuration from ahriman.core.configuration import Configuration
from ahriman.core.exceptions import BuildFailed from ahriman.core.exceptions import BuildFailed
@ -32,37 +32,39 @@ class GPG:
""" """
gnupg wrapper gnupg wrapper
:ivar architecture: repository architecture :ivar architecture: repository architecture
:ivar config: configuration instance :ivar configuration: configuration instance
:ivar default_key: default PGP key ID to use :ivar default_key: default PGP key ID to use
:ivar logger: class logger :ivar logger: class logger
:ivar target: list of targets to sign (repository, package etc) :ivar targets: list of targets to sign (repository, package etc)
""" """
_check_output = check_output _check_output = check_output
def __init__(self, architecture: str, config: Configuration) -> None: def __init__(self, architecture: str, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
self.logger = logging.getLogger("build_details") self.logger = logging.getLogger("build_details")
self.config = config self.architecture = architecture
self.section = config.get_section_name("sign", architecture) self.configuration = configuration
self.target = {SignSettings.from_option(opt) for opt in config.getlist(self.section, "target")} self.targets, self.default_key = self.sign_options(configuration)
self.default_key = config.get(self.section, "key") if self.target else ""
@property @property
def repository_sign_args(self) -> List[str]: def repository_sign_args(self) -> List[str]:
""" """
:return: command line arguments for repo-add command to sign database :return: command line arguments for repo-add command to sign database
""" """
if SignSettings.SignRepository not in self.target: if SignSettings.SignRepository not in self.targets:
return []
if self.default_key is None:
self.logger.error("no default key set, skip repository sign")
return [] return []
return ["--sign", "--key", self.default_key] return ["--sign", "--key", self.default_key]
@staticmethod @staticmethod
def sign_cmd(path: Path, key: str) -> List[str]: def sign_command(path: Path, key: str) -> List[str]:
""" """
gpg command to run gpg command to run
:param path: path to file to sign :param path: path to file to sign
@ -71,6 +73,20 @@ class GPG:
""" """
return ["gpg", "-u", key, "-b", str(path)] return ["gpg", "-u", key, "-b", str(path)]
@staticmethod
def sign_options(configuration: Configuration) -> Tuple[Set[SignSettings], Optional[str]]:
"""
extract default sign options from configuration
:param configuration: configuration instance
:return: tuple of sign targets and default PGP key
"""
targets = {
SignSettings.from_option(option)
for option in configuration.getlist("sign", "target")
}
default_key = configuration.get("sign", "key") if targets else None
return targets, default_key
def process(self, path: Path, key: str) -> List[Path]: def process(self, path: Path, key: str) -> List[Path]:
""" """
gpg command wrapper gpg command wrapper
@ -79,7 +95,7 @@ class GPG:
:return: list of generated files including original file :return: list of generated files including original file
""" """
GPG._check_output( GPG._check_output(
*GPG.sign_cmd(path, key), *GPG.sign_command(path, key),
exception=BuildFailed(path.name), exception=BuildFailed(path.name),
logger=self.logger) logger=self.logger)
return [path, path.parent / f"{path.name}.sig"] return [path, path.parent / f"{path.name}.sig"]
@ -91,9 +107,12 @@ class GPG:
:param base: package base required to check for key overrides :param base: package base required to check for key overrides
:return: list of generated files including original file :return: list of generated files including original file
""" """
if SignSettings.SignPackages not in self.target: if SignSettings.SignPackages not in self.targets:
return [path]
key = self.configuration.get("sign", f"key_{base}", fallback=self.default_key)
if key is None:
self.logger.error(f"no default key set, skip package {path} sign")
return [path] return [path]
key = self.config.get(self.section, f"key_{base}", fallback=self.default_key)
return self.process(path, key) return self.process(path, key)
def sign_repository(self, path: Path) -> List[Path]: def sign_repository(self, path: Path) -> List[Path]:
@ -103,6 +122,9 @@ class GPG:
:param path: path to repository database :param path: path to repository database
:return: list of generated files including original file :return: list of generated files including original file
""" """
if SignSettings.SignRepository not in self.target: if SignSettings.SignRepository not in self.targets:
return [path]
if self.default_key is None:
self.logger.error("no default key set, skip repository sign")
return [path] return [path]
return self.process(path, self.default_key) return self.process(path, self.default_key)

View File

@ -19,7 +19,7 @@
# #
from __future__ import annotations from __future__ import annotations
from typing import List, Optional, Tuple from typing import List, Optional, Tuple, Type
from ahriman.core.configuration import Configuration from ahriman.core.configuration import Configuration
from ahriman.models.build_status import BuildStatus, BuildStatusEnum from ahriman.models.build_status import BuildStatus, BuildStatusEnum
@ -31,6 +31,20 @@ class Client:
base build status reporter client base build status reporter client
""" """
@classmethod
def load(cls: Type[Client], configuration: Configuration) -> Client:
"""
load client from settings
:param configuration: configuration instance
:return: client according to current settings
"""
host = configuration.get("web", "host", fallback=None)
port = configuration.getint("web", "port", fallback=None)
if host is not None and port is not None:
from ahriman.core.status.web_client import WebClient
return WebClient(host, port)
return cls()
def add(self, package: Package, status: BuildStatusEnum) -> None: def add(self, package: Package, status: BuildStatusEnum) -> None:
""" """
add new package with status add new package with status
@ -109,20 +123,3 @@ class Client:
:param package: current package properties :param package: current package properties
""" """
return self.add(package, BuildStatusEnum.Unknown) return self.add(package, BuildStatusEnum.Unknown)
@staticmethod
def load(architecture: str, config: Configuration) -> Client:
"""
load client from settings
:param architecture: repository architecture
:param config: configuration instance
:return: client according to current settings
"""
section = config.get_section_name("web", architecture)
host = config.get(section, "host", fallback=None)
port = config.getint(section, "port", fallback=None)
if host is None or port is None:
return Client()
from ahriman.core.status.web_client import WebClient
return WebClient(host, port)

View File

@ -40,16 +40,16 @@ class Watcher:
:ivar status: daemon status :ivar status: daemon status
""" """
def __init__(self, architecture: str, config: Configuration) -> None: def __init__(self, architecture: str, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
self.logger = logging.getLogger("http") self.logger = logging.getLogger("http")
self.architecture = architecture self.architecture = architecture
self.repository = Repository(architecture, config) self.repository = Repository(architecture, configuration)
self.known: Dict[str, Tuple[Package, BuildStatus]] = {} self.known: Dict[str, Tuple[Package, BuildStatus]] = {}
self.status = BuildStatus() self.status = BuildStatus()

View File

@ -20,41 +20,32 @@
from pathlib import Path from pathlib import Path
from ahriman.core.configuration import Configuration from ahriman.core.configuration import Configuration
from ahriman.core.upload.uploader import Uploader from ahriman.core.upload.upload import Upload
from ahriman.core.util import check_output from ahriman.core.util import check_output
class Rsync(Uploader): class Rsync(Upload):
""" """
rsync wrapper rsync wrapper
:ivar command: command arguments for sync
:ivar remote: remote address to sync :ivar remote: remote address to sync
""" """
_check_output = check_output _check_output = check_output
def __init__(self, architecture: str, config: Configuration) -> None: def __init__(self, architecture: str, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
Uploader.__init__(self, architecture, config) Upload.__init__(self, architecture, configuration)
section = config.get_section_name("rsync", architecture) self.command = configuration.getlist("rsync", "command")
self.remote = config.get(section, "remote") self.remote = configuration.get("rsync", "remote")
def sync(self, path: Path) -> None: def sync(self, path: Path) -> None:
""" """
sync data to remote server sync data to remote server
:param path: local path to sync :param path: local path to sync
""" """
Rsync._check_output( Rsync._check_output(*self.command, str(path), self.remote, exception=None, logger=self.logger)
"rsync",
"--archive",
"--verbose",
"--compress",
"--partial",
"--delete",
str(path),
self.remote,
exception=None,
logger=self.logger)

View File

@ -20,27 +20,28 @@
from pathlib import Path from pathlib import Path
from ahriman.core.configuration import Configuration from ahriman.core.configuration import Configuration
from ahriman.core.upload.uploader import Uploader from ahriman.core.upload.upload import Upload
from ahriman.core.util import check_output from ahriman.core.util import check_output
class S3(Uploader): class S3(Upload):
""" """
aws-cli wrapper aws-cli wrapper
:ivar bucket: full bucket name :ivar bucket: full bucket name
:ivar command: command arguments for sync
""" """
_check_output = check_output _check_output = check_output
def __init__(self, architecture: str, config: Configuration) -> None: def __init__(self, architecture: str, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
Uploader.__init__(self, architecture, config) Upload.__init__(self, architecture, configuration)
section = config.get_section_name("s3", architecture) self.bucket = configuration.get("s3", "bucket")
self.bucket = config.get(section, "bucket") self.command = configuration.getlist("s3", "command")
def sync(self, path: Path) -> None: def sync(self, path: Path) -> None:
""" """
@ -48,6 +49,4 @@ class S3(Uploader):
:param path: local path to sync :param path: local path to sync
""" """
# TODO rewrite to boto, but it is bullshit # TODO rewrite to boto, but it is bullshit
S3._check_output("aws", "s3", "sync", "--quiet", "--delete", str(path), self.bucket, S3._check_output(*self.command, str(path), self.bucket, exception=None, logger=self.logger)
exception=None,
logger=self.logger)

View File

@ -17,56 +17,63 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
# #
from __future__ import annotations
import logging import logging
from pathlib import Path from pathlib import Path
from typing import Type
from ahriman.core.configuration import Configuration from ahriman.core.configuration import Configuration
from ahriman.core.exceptions import SyncFailed from ahriman.core.exceptions import SyncFailed
from ahriman.models.upload_settings import UploadSettings from ahriman.models.upload_settings import UploadSettings
class Uploader: class Upload:
""" """
base remote sync class base remote sync class
:ivar architecture: repository architecture :ivar architecture: repository architecture
:ivar config: configuration instance :ivar configuration: configuration instance
:ivar logger: application logger :ivar logger: application logger
""" """
def __init__(self, architecture: str, config: Configuration) -> None: def __init__(self, architecture: str, configuration: Configuration) -> None:
""" """
default constructor default constructor
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
""" """
self.logger = logging.getLogger("builder") self.logger = logging.getLogger("builder")
self.architecture = architecture self.architecture = architecture
self.config = config self.config = configuration
@staticmethod @classmethod
def run(architecture: str, config: Configuration, target: str, path: Path) -> None: def load(cls: Type[Upload], architecture: str, configuration: Configuration, target: str) -> Upload:
""" """
run remote sync load client from settings
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
:param target: target to run sync (e.g. s3) :param target: target to run sync (e.g. s3)
:param path: local path to sync :return: client according to current settings
""" """
provider = UploadSettings.from_option(target) provider = UploadSettings.from_option(target)
if provider == UploadSettings.Rsync: if provider == UploadSettings.Rsync:
from ahriman.core.upload.rsync import Rsync from ahriman.core.upload.rsync import Rsync
uploader: Uploader = Rsync(architecture, config) return Rsync(architecture, configuration)
elif provider == UploadSettings.S3: if provider == UploadSettings.S3:
from ahriman.core.upload.s3 import S3 from ahriman.core.upload.s3 import S3
uploader = S3(architecture, config) return S3(architecture, configuration)
else: return cls(architecture, configuration) # should never happen
uploader = Uploader(architecture, config)
def run(self, path: Path) -> None:
"""
run remote sync
:param path: local path to sync
"""
try: try:
uploader.sync(path) self.sync(path)
except Exception: except Exception:
uploader.logger.exception(f"remote sync failed for {provider.name}") self.logger.exception("remote sync failed")
raise SyncFailed() raise SyncFailed()
def sync(self, path: Path) -> None: def sync(self, path: Path) -> None:

View File

@ -63,7 +63,7 @@ class BuildStatus:
""" """
build status holder build status holder
:ivar status: build status :ivar status: build status
:ivar _timestamp: build status update time :ivar timestamp: build status update time
""" """
def __init__(self, status: Union[BuildStatusEnum, str, None] = None, def __init__(self, status: Union[BuildStatusEnum, str, None] = None,

View File

@ -31,7 +31,7 @@ from typing import Any, Dict, List, Optional, Set, Type, Union
from ahriman.core.alpm.pacman import Pacman from ahriman.core.alpm.pacman import Pacman
from ahriman.core.exceptions import InvalidPackageInfo from ahriman.core.exceptions import InvalidPackageInfo
from ahriman.core.util import check_output from ahriman.core.util import check_output
from ahriman.models.package_desciption import PackageDescription from ahriman.models.package_description import PackageDescription
from ahriman.models.repository_paths import RepositoryPaths from ahriman.models.repository_paths import RepositoryPaths
@ -59,6 +59,13 @@ class Package:
""" """
return f"{self.aur_url}/{self.base}.git" return f"{self.aur_url}/{self.base}.git"
@property
def groups(self) -> List[str]:
"""
:return: sum of groups per each package
"""
return sorted(set(sum([package.groups for package in self.packages.values()], start=[])))
@property @property
def is_single_package(self) -> bool: def is_single_package(self) -> bool:
""" """
@ -78,6 +85,13 @@ class Package:
or self.base.endswith("-hg")\ or self.base.endswith("-hg")\
or self.base.endswith("-svn") or self.base.endswith("-svn")
@property
def licenses(self) -> List[str]:
"""
:return: sum of licenses per each package
"""
return sorted(set(sum([package.licenses for package in self.packages.values()], start=[])))
@property @property
def web_url(self) -> str: def web_url(self) -> str:
""" """
@ -95,8 +109,8 @@ class Package:
:return: package properties :return: package properties
""" """
package = pacman.handle.load_pkg(str(path)) package = pacman.handle.load_pkg(str(path))
properties = PackageDescription(package.size, package.builddate, path.name, package.isize) return cls(package.base, package.version, aur_url,
return cls(package.base, package.version, aur_url, {package.name: properties}) {package.name: PackageDescription.from_package(package, path)})
@classmethod @classmethod
def from_aur(cls: Type[Package], name: str, aur_url: str) -> Package: def from_aur(cls: Type[Package], name: str, aur_url: str) -> Package:
@ -142,6 +156,27 @@ class Package:
aur_url=dump["aur_url"], aur_url=dump["aur_url"],
packages=packages) packages=packages)
@classmethod
def load(cls: Type[Package], path: Union[Path, str], pacman: Pacman, aur_url: str) -> Package:
"""
package constructor from available sources
:param path: one of path to sources directory, path to archive or package name/base
:param pacman: alpm wrapper instance (required to load from archive)
:param aur_url: AUR root url
:return: package properties
"""
try:
maybe_path = Path(path)
if maybe_path.is_dir():
return cls.from_build(maybe_path, aur_url)
if maybe_path.is_file():
return cls.from_archive(maybe_path, pacman, aur_url)
return cls.from_aur(str(path), aur_url)
except InvalidPackageInfo:
raise
except Exception as e:
raise InvalidPackageInfo(str(e))
@staticmethod @staticmethod
def dependencies(path: Path) -> Set[str]: def dependencies(path: Path) -> Set[str]:
""" """
@ -180,29 +215,6 @@ class Package:
prefix = f"{epoch}:" if epoch else "" prefix = f"{epoch}:" if epoch else ""
return f"{prefix}{pkgver}-{pkgrel}" return f"{prefix}{pkgver}-{pkgrel}"
@staticmethod
def load(path: Union[Path, str], pacman: Pacman, aur_url: str) -> Package:
"""
package constructor from available sources
:param path: one of path to sources directory, path to archive or package name/base
:param pacman: alpm wrapper instance (required to load from archive)
:param aur_url: AUR root url
:return: package properties
"""
try:
maybe_path = Path(path)
if maybe_path.is_dir():
package: Package = Package.from_build(maybe_path, aur_url)
elif maybe_path.is_file():
package = Package.from_archive(maybe_path, pacman, aur_url)
else:
package = Package.from_aur(str(path), aur_url)
return package
except InvalidPackageInfo:
raise
except Exception as e:
raise InvalidPackageInfo(str(e))
def actual_version(self, paths: RepositoryPaths) -> str: def actual_version(self, paths: RepositoryPaths) -> str:
""" """
additional method to handle VCS package versions additional method to handle VCS package versions

View File

@ -17,25 +17,38 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
# #
from dataclasses import dataclass from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path from pathlib import Path
from typing import Optional from pyalpm import Package # type: ignore
from typing import List, Optional, Type
@dataclass @dataclass
class PackageDescription: class PackageDescription:
""" """
package specific properties package specific properties
:ivar architecture: package architecture
:ivar archive_size: package archive size :ivar archive_size: package archive size
:ivar build_date: package build date :ivar build_date: package build date
:ivar description: package description
:ivar filename: package archive name :ivar filename: package archive name
:ivar groups: package groups
:ivar installed_size: package installed size :ivar installed_size: package installed size
:ivar licenses: package licenses list
:ivar url: package url
""" """
architecture: Optional[str] = None
archive_size: Optional[int] = None archive_size: Optional[int] = None
build_date: Optional[int] = None build_date: Optional[int] = None
description: Optional[str] = None
filename: Optional[str] = None filename: Optional[str] = None
groups: List[str] = field(default_factory=list)
installed_size: Optional[int] = None installed_size: Optional[int] = None
licenses: List[str] = field(default_factory=list)
url: Optional[str] = None
@property @property
def filepath(self) -> Optional[Path]: def filepath(self) -> Optional[Path]:
@ -43,3 +56,22 @@ class PackageDescription:
:return: path object for current filename :return: path object for current filename
""" """
return Path(self.filename) if self.filename is not None else None return Path(self.filename) if self.filename is not None else None
@classmethod
def from_package(cls: Type[PackageDescription], package: Package, path: Path) -> PackageDescription:
"""
construct class from alpm package class
:param package: alpm generated object
:param path: path to package archive
:return: package properties based on tarball
"""
return cls(
architecture=package.arch,
archive_size=package.size,
build_date=package.builddate,
description=package.desc,
filename=path.name,
groups=package.groups,
installed_size=package.isize,
licenses=package.licenses,
url=package.url)

View File

@ -20,6 +20,7 @@
from __future__ import annotations from __future__ import annotations
from enum import Enum, auto from enum import Enum, auto
from typing import Type
from ahriman.core.exceptions import InvalidOption from ahriman.core.exceptions import InvalidOption
@ -32,13 +33,13 @@ class ReportSettings(Enum):
HTML = auto() HTML = auto()
@staticmethod @classmethod
def from_option(value: str) -> ReportSettings: def from_option(cls: Type[ReportSettings], value: str) -> ReportSettings:
""" """
construct value from configuration construct value from configuration
:param value: configuration value :param value: configuration value
:return: parsed value :return: parsed value
""" """
if value.lower() in ("html",): if value.lower() in ("html",):
return ReportSettings.HTML return cls.HTML
raise InvalidOption(value) raise InvalidOption(value)

View File

@ -20,6 +20,7 @@
from __future__ import annotations from __future__ import annotations
from enum import Enum, auto from enum import Enum, auto
from typing import Type
from ahriman.core.exceptions import InvalidOption from ahriman.core.exceptions import InvalidOption
@ -34,15 +35,15 @@ class SignSettings(Enum):
SignPackages = auto() SignPackages = auto()
SignRepository = auto() SignRepository = auto()
@staticmethod @classmethod
def from_option(value: str) -> SignSettings: def from_option(cls: Type[SignSettings], value: str) -> SignSettings:
""" """
construct value from configuration construct value from configuration
:param value: configuration value :param value: configuration value
:return: parsed value :return: parsed value
""" """
if value.lower() in ("package", "packages", "sign-package"): if value.lower() in ("package", "packages", "sign-package"):
return SignSettings.SignPackages return cls.SignPackages
if value.lower() in ("repository", "sign-repository"): if value.lower() in ("repository", "sign-repository"):
return SignSettings.SignRepository return cls.SignRepository
raise InvalidOption(value) raise InvalidOption(value)

View File

@ -20,6 +20,7 @@
from __future__ import annotations from __future__ import annotations
from enum import Enum, auto from enum import Enum, auto
from typing import Type
from ahriman.core.exceptions import InvalidOption from ahriman.core.exceptions import InvalidOption
@ -34,15 +35,15 @@ class UploadSettings(Enum):
Rsync = auto() Rsync = auto()
S3 = auto() S3 = auto()
@staticmethod @classmethod
def from_option(value: str) -> UploadSettings: def from_option(cls: Type[UploadSettings], value: str) -> UploadSettings:
""" """
construct value from configuration construct value from configuration
:param value: configuration value :param value: configuration value
:return: parsed value :return: parsed value
""" """
if value.lower() in ("rsync",): if value.lower() in ("rsync",):
return UploadSettings.Rsync return cls.Rsync
if value.lower() in ("s3",): if value.lower() in ("s3",):
return UploadSettings.S3 return cls.S3
raise InvalidOption(value) raise InvalidOption(value)

View File

@ -17,4 +17,4 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
# #
__version__ = "0.17.0" __version__ = "0.20.0"

View File

@ -34,10 +34,20 @@ class IndexView(BaseView):
It uses jinja2 templates for report generation, the following variables are allowed: It uses jinja2 templates for report generation, the following variables are allowed:
architecture - repository architecture, string, required architecture - repository architecture, string, required
packages - sorted list of packages properties: base, packages (sorted list), status, packages - sorted list of packages properties, required
timestamp, version, web_url. Required * base, string
* groups, sorted list of strings
* licenses, sorted list of strings
* packages, sorted list of strings
* status, string based on enum value
* timestamp, pretty printed datetime, string
* version, string
* web_url, string
repository - repository name, string, required repository - repository name, string, required
service - service status properties: status, status_color, timestamp. Required service - service status properties, required
* status, string based on enum value
* status_color, string based on enum value
* timestamp, pretty printed datetime, string
version - ahriman version, string, required version - ahriman version, string, required
""" """
@ -51,6 +61,8 @@ class IndexView(BaseView):
packages = [ packages = [
{ {
"base": package.base, "base": package.base,
"groups": package.groups,
"licenses": package.licenses,
"packages": list(sorted(package.packages)), "packages": list(sorted(package.packages)),
"status": status.status.value, "status": status.status.value,
"timestamp": pretty_datetime(status.timestamp), "timestamp": pretty_datetime(status.timestamp),

View File

@ -58,19 +58,19 @@ def run_server(application: web.Application) -> None:
""" """
application.logger.info("start server") application.logger.info("start server")
section = application["config"].get_section_name("web", application["architecture"]) configuration: Configuration = application["configuration"]
host = application["config"].get(section, "host") host = configuration.get("web", "host")
port = application["config"].getint(section, "port") port = configuration.getint("web", "port")
web.run_app(application, host=host, port=port, handle_signals=False, web.run_app(application, host=host, port=port, handle_signals=False,
access_log=logging.getLogger("http")) access_log=logging.getLogger("http"))
def setup_service(architecture: str, config: Configuration) -> web.Application: def setup_service(architecture: str, configuration: Configuration) -> web.Application:
""" """
create web application create web application
:param architecture: repository architecture :param architecture: repository architecture
:param config: configuration instance :param configuration: configuration instance
:return: web application instance :return: web application instance
""" """
application = web.Application(logger=logging.getLogger("http")) application = web.Application(logger=logging.getLogger("http"))
@ -84,13 +84,12 @@ def setup_service(architecture: str, config: Configuration) -> web.Application:
setup_routes(application) setup_routes(application)
application.logger.info("setup templates") application.logger.info("setup templates")
aiohttp_jinja2.setup(application, loader=jinja2.FileSystemLoader(config.getpath("web", "templates"))) aiohttp_jinja2.setup(application, loader=jinja2.FileSystemLoader(configuration.getpath("web", "templates")))
application.logger.info("setup configuration") application.logger.info("setup configuration")
application["config"] = config application["configuration"] = configuration
application["architecture"] = architecture
application.logger.info("setup watcher") application.logger.info("setup watcher")
application["watcher"] = Watcher(architecture, config) application["watcher"] = Watcher(architecture, configuration)
return application return application

View File

@ -3,25 +3,27 @@ import argparse
from pytest_mock import MockerFixture from pytest_mock import MockerFixture
from ahriman.application.handlers import Handler from ahriman.application.handlers import Handler
from ahriman.core.configuration import Configuration
def test_call(args: argparse.Namespace, configuration: Configuration, mocker: MockerFixture) -> None: def test_call(args: argparse.Namespace, mocker: MockerFixture) -> None:
""" """
must call inside lock must call inside lock
""" """
args.configuration = ""
args.no_log = False
mocker.patch("ahriman.application.handlers.Handler.run") mocker.patch("ahriman.application.handlers.Handler.run")
mocker.patch("ahriman.core.configuration.Configuration.from_path")
enter_mock = mocker.patch("ahriman.application.lock.Lock.__enter__") enter_mock = mocker.patch("ahriman.application.lock.Lock.__enter__")
exit_mock = mocker.patch("ahriman.application.lock.Lock.__exit__") exit_mock = mocker.patch("ahriman.application.lock.Lock.__exit__")
assert Handler._call(args, "x86_64", configuration) assert Handler._call(args, "x86_64")
enter_mock.assert_called_once() enter_mock.assert_called_once()
exit_mock.assert_called_once() exit_mock.assert_called_once()
def test_call_exception(args: argparse.Namespace, configuration: Configuration, mocker: MockerFixture) -> None: def test_call_exception(args: argparse.Namespace, mocker: MockerFixture) -> None:
""" """
must process exception must process exception
""" """
mocker.patch("ahriman.application.lock.Lock.__enter__", side_effect=Exception()) mocker.patch("ahriman.application.lock.Lock.__enter__", side_effect=Exception())
assert not Handler._call(args, "x86_64", configuration) assert not Handler._call(args, "x86_64")

View File

@ -8,6 +8,7 @@ from ahriman.core.configuration import Configuration
def _default_args(args: argparse.Namespace) -> argparse.Namespace: def _default_args(args: argparse.Namespace) -> argparse.Namespace:
args.package = [] args.package = []
args.now = False
args.without_dependencies = False args.without_dependencies = False
return args return args
@ -22,3 +23,19 @@ def test_run(args: argparse.Namespace, configuration: Configuration, mocker: Moc
Add.run(args, "x86_64", configuration) Add.run(args, "x86_64", configuration)
application_mock.assert_called_once() application_mock.assert_called_once()
def test_run_with_updates(args: argparse.Namespace, configuration: Configuration, mocker: MockerFixture) -> None:
"""
must run command with updates after
"""
args = _default_args(args)
args.now = True
mocker.patch("pathlib.Path.mkdir")
mocker.patch("ahriman.application.application.Application.add")
application_mock = mocker.patch("ahriman.application.application.Application.update")
updates_mock = mocker.patch("ahriman.application.application.Application.get_updates")
Add.run(args, "x86_64", configuration)
application_mock.assert_called_once()
updates_mock.assert_called_once()

View File

@ -11,7 +11,10 @@ def test_run(args: argparse.Namespace, configuration: Configuration, mocker: Moc
must run command must run command
""" """
mocker.patch("pathlib.Path.mkdir") mocker.patch("pathlib.Path.mkdir")
application_mock = mocker.patch("ahriman.core.configuration.Configuration.dump") print_mock = mocker.patch("ahriman.application.handlers.dump.Dump._print")
application_mock = mocker.patch("ahriman.core.configuration.Configuration.dump",
return_value=configuration.dump())
Dump.run(args, "x86_64", configuration) Dump.run(args, "x86_64", configuration)
application_mock.assert_called_once() application_mock.assert_called_once()
print_mock.assert_called()

View File

@ -11,7 +11,7 @@ from ahriman.models.repository_paths import RepositoryPaths
def _default_args(args: argparse.Namespace) -> argparse.Namespace: def _default_args(args: argparse.Namespace) -> argparse.Namespace:
args.build_command = "ahriman" args.build_command = "ahriman"
args.from_config = "/usr/share/devtools/pacman-extra.conf" args.from_configuration = "/usr/share/devtools/pacman-extra.conf"
args.no_multilib = False args.no_multilib = False
args.packager = "John Doe <john@doe.com>" args.packager = "John Doe <john@doe.com>"
args.repository = "aur-clone" args.repository = "aur-clone"
@ -81,8 +81,8 @@ def test_create_devtools_configuration(args: argparse.Namespace, repository_path
add_section_mock = mocker.patch("configparser.RawConfigParser.add_section") add_section_mock = mocker.patch("configparser.RawConfigParser.add_section")
write_mock = mocker.patch("configparser.RawConfigParser.write") write_mock = mocker.patch("configparser.RawConfigParser.write")
Setup.create_devtools_configuration(args.build_command, "x86_64", Path(args.from_config), args.no_multilib, Setup.create_devtools_configuration(args.build_command, "x86_64", Path(args.from_configuration),
args.repository, repository_paths) args.no_multilib, args.repository, repository_paths)
add_section_mock.assert_has_calls([ add_section_mock.assert_has_calls([
mock.call("multilib"), mock.call("multilib"),
mock.call(args.repository) mock.call(args.repository)
@ -101,8 +101,8 @@ def test_create_devtools_configuration_no_multilib(args: argparse.Namespace, rep
add_section_mock = mocker.patch("configparser.RawConfigParser.add_section") add_section_mock = mocker.patch("configparser.RawConfigParser.add_section")
write_mock = mocker.patch("configparser.RawConfigParser.write") write_mock = mocker.patch("configparser.RawConfigParser.write")
Setup.create_devtools_configuration(args.build_command, "x86_64", Path(args.from_config), True, Setup.create_devtools_configuration(args.build_command, "x86_64", Path(args.from_configuration),
args.repository, repository_paths) True, args.repository, repository_paths)
add_section_mock.assert_called_once() add_section_mock.assert_called_once()
write_mock.assert_called_once() write_mock.assert_called_once()

View File

@ -0,0 +1,40 @@
import argparse
from pytest_mock import MockerFixture
from ahriman.application.handlers import StatusUpdate
from ahriman.core.configuration import Configuration
from ahriman.models.build_status import BuildStatusEnum
from ahriman.models.package import Package
def _default_args(args: argparse.Namespace) -> argparse.Namespace:
args.status = BuildStatusEnum.Success.value
args.package = None
return args
def test_run(args: argparse.Namespace, configuration: Configuration, mocker: MockerFixture) -> None:
"""
must run command
"""
args = _default_args(args)
mocker.patch("pathlib.Path.mkdir")
update_self_mock = mocker.patch("ahriman.core.status.client.Client.update_self")
StatusUpdate.run(args, "x86_64", configuration)
update_self_mock.assert_called_once()
def test_run_packages(args: argparse.Namespace, configuration: Configuration, package_ahriman: Package,
mocker: MockerFixture) -> None:
"""
must run command with specified packages
"""
args = _default_args(args)
args.package = [package_ahriman.base]
mocker.patch("pathlib.Path.mkdir")
update_mock = mocker.patch("ahriman.core.status.client.Client.update")
StatusUpdate.run(args, "x86_64", configuration)
update_mock.assert_called_once()

View File

@ -64,6 +64,16 @@ def test_subparsers_status(parser: argparse.ArgumentParser) -> None:
assert args.unsafe assert args.unsafe
def test_subparsers_status_update(parser: argparse.ArgumentParser) -> None:
"""
status-update command must imply lock, no_report and unsafe
"""
args = parser.parse_args(["-a", "x86_64", "status-update"])
assert args.lock is None
assert args.no_report
assert args.unsafe
def test_subparsers_web(parser: argparse.ArgumentParser) -> None: def test_subparsers_web(parser: argparse.ArgumentParser) -> None:
""" """
web command must imply lock and no_report web command must imply lock and no_report

View File

@ -15,14 +15,14 @@ def test_enter(lock: Lock, mocker: MockerFixture) -> None:
must process with context manager must process with context manager
""" """
check_user_mock = mocker.patch("ahriman.application.lock.Lock.check_user") check_user_mock = mocker.patch("ahriman.application.lock.Lock.check_user")
remove_mock = mocker.patch("ahriman.application.lock.Lock.remove") clear_mock = mocker.patch("ahriman.application.lock.Lock.clear")
create_mock = mocker.patch("ahriman.application.lock.Lock.create") create_mock = mocker.patch("ahriman.application.lock.Lock.create")
update_status_mock = mocker.patch("ahriman.core.status.client.Client.update_self") update_status_mock = mocker.patch("ahriman.core.status.client.Client.update_self")
with lock: with lock:
pass pass
check_user_mock.assert_called_once() check_user_mock.assert_called_once()
remove_mock.assert_called_once() clear_mock.assert_called_once()
create_mock.assert_called_once() create_mock.assert_called_once()
update_status_mock.assert_has_calls([ update_status_mock.assert_has_calls([
mock.call(BuildStatusEnum.Building), mock.call(BuildStatusEnum.Building),
@ -35,7 +35,7 @@ def test_exit_with_exception(lock: Lock, mocker: MockerFixture) -> None:
must process with context manager in case if exception raised must process with context manager in case if exception raised
""" """
mocker.patch("ahriman.application.lock.Lock.check_user") mocker.patch("ahriman.application.lock.Lock.check_user")
mocker.patch("ahriman.application.lock.Lock.remove") mocker.patch("ahriman.application.lock.Lock.clear")
mocker.patch("ahriman.application.lock.Lock.create") mocker.patch("ahriman.application.lock.Lock.create")
update_status_mock = mocker.patch("ahriman.core.status.client.Client.update_self") update_status_mock = mocker.patch("ahriman.core.status.client.Client.update_self")
@ -79,6 +79,34 @@ def test_check_user_unsafe(lock: Lock) -> None:
lock.check_user() lock.check_user()
def test_clear(lock: Lock) -> None:
"""
must remove lock file
"""
lock.path = Path(tempfile.mktemp())
lock.path.touch()
lock.clear()
assert not lock.path.is_file()
def test_clear_missing(lock: Lock) -> None:
"""
must not fail on lock removal if file is missing
"""
lock.path = Path(tempfile.mktemp())
lock.clear()
def test_clear_skip(lock: Lock, mocker: MockerFixture) -> None:
"""
must skip removal if no file set
"""
unlink_mock = mocker.patch("pathlib.Path.unlink")
lock.clear()
unlink_mock.assert_not_called()
def test_create(lock: Lock) -> None: def test_create(lock: Lock) -> None:
""" """
must create lock must create lock
@ -121,31 +149,3 @@ def test_create_unsafe(lock: Lock) -> None:
lock.create() lock.create()
lock.path.unlink() lock.path.unlink()
def test_remove(lock: Lock) -> None:
"""
must remove lock file
"""
lock.path = Path(tempfile.mktemp())
lock.path.touch()
lock.remove()
assert not lock.path.is_file()
def test_remove_missing(lock: Lock) -> None:
"""
must not fail on lock removal if file is missing
"""
lock.path = Path(tempfile.mktemp())
lock.remove()
def test_remove_skip(lock: Lock, mocker: MockerFixture) -> None:
"""
must skip removal if no file set
"""
unlink_mock = mocker.patch("pathlib.Path.unlink")
lock.remove()
unlink_mock.assert_not_called()

View File

@ -7,7 +7,7 @@ from typing import Any, Type, TypeVar
from ahriman.core.configuration import Configuration from ahriman.core.configuration import Configuration
from ahriman.core.status.watcher import Watcher from ahriman.core.status.watcher import Watcher
from ahriman.models.package import Package from ahriman.models.package import Package
from ahriman.models.package_desciption import PackageDescription from ahriman.models.package_description import PackageDescription
from ahriman.models.repository_paths import RepositoryPaths from ahriman.models.repository_paths import RepositoryPaths
T = TypeVar("T") T = TypeVar("T")
@ -27,7 +27,7 @@ def anyvar(cls: Type[T], strict: bool = False) -> T:
@pytest.fixture @pytest.fixture
def configuration(resource_path_root: Path) -> Configuration: def configuration(resource_path_root: Path) -> Configuration:
path = resource_path_root / "core" / "ahriman.ini" path = resource_path_root / "core" / "ahriman.ini"
return Configuration.from_path(path=path, logfile=False) return Configuration.from_path(path=path, architecture="x86_64", logfile=False)
@pytest.fixture @pytest.fixture
@ -58,28 +58,43 @@ def package_python_schedule(
@pytest.fixture @pytest.fixture
def package_description_ahriman() -> PackageDescription: def package_description_ahriman() -> PackageDescription:
return PackageDescription( return PackageDescription(
architecture="x86_64",
archive_size=4200, archive_size=4200,
build_date=42, build_date=42,
description="ArcHlinux ReposItory MANager",
filename="ahriman-0.12.1-1-any.pkg.tar.zst", filename="ahriman-0.12.1-1-any.pkg.tar.zst",
installed_size=4200000) groups=[],
installed_size=4200000,
licenses=["GPL3"],
url="https://github.com/arcan1s/ahriman")
@pytest.fixture @pytest.fixture
def package_description_python_schedule() -> PackageDescription: def package_description_python_schedule() -> PackageDescription:
return PackageDescription( return PackageDescription(
architecture="x86_64",
archive_size=4201, archive_size=4201,
build_date=421, build_date=421,
description="Python job scheduling for humans.",
filename="python-schedule-1.0.0-2-any.pkg.tar.zst", filename="python-schedule-1.0.0-2-any.pkg.tar.zst",
installed_size=4200001) groups=[],
installed_size=4200001,
licenses=["MIT"],
url="https://github.com/dbader/schedule")
@pytest.fixture @pytest.fixture
def package_description_python2_schedule() -> PackageDescription: def package_description_python2_schedule() -> PackageDescription:
return PackageDescription( return PackageDescription(
architecture="x86_64",
archive_size=4202, archive_size=4202,
build_date=422, build_date=422,
description="Python job scheduling for humans.",
filename="python2-schedule-1.0.0-2-any.pkg.tar.zst", filename="python2-schedule-1.0.0-2-any.pkg.tar.zst",
installed_size=4200002) groups=[],
installed_size=4200002,
licenses=["MIT"],
url="https://github.com/dbader/schedule")
@pytest.fixture @pytest.fixture

View File

@ -1,5 +1,4 @@
import pytest import pytest
import shutil
from pathlib import Path from pathlib import Path
from pytest_mock import MockerFixture from pytest_mock import MockerFixture

View File

@ -31,4 +31,4 @@ def repo(configuration: Configuration, repository_paths: RepositoryPaths) -> Rep
@pytest.fixture @pytest.fixture
def task_ahriman(package_ahriman: Package, configuration: Configuration, repository_paths: RepositoryPaths) -> Task: def task_ahriman(package_ahriman: Package, configuration: Configuration, repository_paths: RepositoryPaths) -> Task:
return Task(package_ahriman, "x86_64", configuration, repository_paths) return Task(package_ahriman, configuration, repository_paths)

View File

@ -0,0 +1,16 @@
from pytest_mock import MockerFixture
from ahriman.core.configuration import Configuration
from ahriman.core.report.html import HTML
from ahriman.models.package import Package
def test_generate(configuration: Configuration, package_ahriman: Package, mocker: MockerFixture) -> None:
"""
must generate report
"""
write_mock = mocker.patch("pathlib.Path.write_text")
report = HTML("x86_64", configuration)
report.generate([package_ahriman])
write_mock.assert_called_once()

View File

@ -0,0 +1,27 @@
import pytest
from pathlib import Path
from pytest_mock import MockerFixture
from ahriman.core.configuration import Configuration
from ahriman.core.exceptions import ReportFailed
from ahriman.core.report.report import Report
from ahriman.models.report_settings import ReportSettings
def test_report_failure(configuration: Configuration, mocker: MockerFixture) -> None:
"""
must raise ReportFailed on errors
"""
mocker.patch("ahriman.core.report.html.HTML.generate", side_effect=Exception())
with pytest.raises(ReportFailed):
Report.load("x86_64", configuration, ReportSettings.HTML.name).run(Path("path"))
def test_report_html(configuration: Configuration, mocker: MockerFixture) -> None:
"""
must generate html report
"""
report_mock = mocker.patch("ahriman.core.report.html.HTML.generate")
Report.load("x86_64", configuration, ReportSettings.HTML.name).run(Path("path"))
report_mock.assert_called_once()

View File

@ -107,20 +107,20 @@ def test_process_report_auto(executor: Executor, mocker: MockerFixture) -> None:
""" """
must process report in auto mode if no targets supplied must process report in auto mode if no targets supplied
""" """
config_getlist_mock = mocker.patch("ahriman.core.configuration.Configuration.getlist") configuration_getlist_mock = mocker.patch("ahriman.core.configuration.Configuration.getlist")
executor.process_report(None) executor.process_report(None)
config_getlist_mock.assert_called_once() configuration_getlist_mock.assert_called_once()
def test_process_sync_auto(executor: Executor, mocker: MockerFixture) -> None: def test_process_sync_auto(executor: Executor, mocker: MockerFixture) -> None:
""" """
must process sync in auto mode if no targets supplied must process sync in auto mode if no targets supplied
""" """
config_getlist_mock = mocker.patch("ahriman.core.configuration.Configuration.getlist") configuration_getlist_mock = mocker.patch("ahriman.core.configuration.Configuration.getlist")
executor.process_sync(None) executor.process_sync(None)
config_getlist_mock.assert_called_once() configuration_getlist_mock.assert_called_once()
def test_process_update(executor: Executor, package_ahriman: Package, mocker: MockerFixture) -> None: def test_process_update(executor: Executor, package_ahriman: Package, mocker: MockerFixture) -> None:

View File

@ -50,7 +50,7 @@ def test_updates_aur_ignore(update_handler: UpdateHandler, package_ahriman: Pack
""" """
must skip ignore packages must skip ignore packages
""" """
mocker.patch("ahriman.core.configuration.Configuration.getlist", return_value=[package_ahriman.base]) update_handler.ignore_list = [package_ahriman.base]
mocker.patch("ahriman.core.repository.update_handler.UpdateHandler.packages", return_value=[package_ahriman]) mocker.patch("ahriman.core.repository.update_handler.UpdateHandler.packages", return_value=[package_ahriman])
package_load_mock = mocker.patch("ahriman.models.package.Package.load") package_load_mock = mocker.patch("ahriman.models.package.Package.load")

View File

@ -7,3 +7,9 @@ from ahriman.core.sign.gpg import GPG
@pytest.fixture @pytest.fixture
def gpg(configuration: Configuration) -> GPG: def gpg(configuration: Configuration) -> GPG:
return GPG("x86_64", configuration) return GPG("x86_64", configuration)
@pytest.fixture
def gpg_with_key(gpg: GPG) -> GPG:
gpg.default_key = "key"
return gpg

View File

@ -1,61 +0,0 @@
from pathlib import Path
from pytest_mock import MockerFixture
from ahriman.core.sign.gpg import GPG
from ahriman.models.sign_settings import SignSettings
def test_repository_sign_args(gpg: GPG) -> None:
"""
must generate correct sign args
"""
gpg.target = {SignSettings.SignRepository}
assert gpg.repository_sign_args
def test_sign_package(gpg: GPG, mocker: MockerFixture) -> None:
"""
must sign package
"""
result = [Path("a"), Path("a.sig")]
process_mock = mocker.patch("ahriman.core.sign.gpg.process", return_value=result)
for target in ({SignSettings.SignPackages}, {SignSettings.SignPackages, SignSettings.SignRepository}):
gpg.target = target
assert gpg.sign_package(Path("a"), "a") == result
process_mock.assert_called_once()
def test_sign_package_skip(gpg: GPG, mocker: MockerFixture) -> None:
"""
must not sign package if it is not set
"""
process_mock = mocker.patch("ahriman.core.sign.gpg.process")
for target in ({}, {SignSettings.SignRepository}):
gpg.target = target
process_mock.assert_not_called()
def test_sign_repository(gpg: GPG, mocker: MockerFixture) -> None:
"""
must sign repository
"""
result = [Path("a"), Path("a.sig")]
process_mock = mocker.patch("ahriman.core.sign.gpg.process", return_value=result)
for target in ({SignSettings.SignRepository}, {SignSettings.SignPackages, SignSettings.SignRepository}):
gpg.target = target
assert gpg.sign_repository(Path("a")) == result
process_mock.assert_called_once()
def test_sign_repository_skip(gpg: GPG, mocker: MockerFixture) -> None:
"""
must not sign repository if it is not set
"""
process_mock = mocker.patch("ahriman.core.sign.gpg.process")
for target in ({}, {SignSettings.SignPackages}):
gpg.target = target
process_mock.assert_not_called()

View File

@ -0,0 +1,181 @@
from pathlib import Path
from pytest_mock import MockerFixture
from ahriman.core.sign.gpg import GPG
from ahriman.models.sign_settings import SignSettings
def test_repository_sign_args_1(gpg_with_key: GPG) -> None:
"""
must generate correct sign args
"""
gpg_with_key.targets = {SignSettings.SignRepository}
assert gpg_with_key.repository_sign_args
def test_repository_sign_args_2(gpg_with_key: GPG) -> None:
"""
must generate correct sign args
"""
gpg_with_key.targets = {SignSettings.SignPackages, SignSettings.SignRepository}
assert gpg_with_key.repository_sign_args
def test_repository_sign_args_skip_1(gpg_with_key: GPG) -> None:
"""
must return empty args if it is not set
"""
gpg_with_key.targets = {}
assert not gpg_with_key.repository_sign_args
def test_repository_sign_args_skip_2(gpg_with_key: GPG) -> None:
"""
must return empty args if it is not set
"""
gpg_with_key.targets = {SignSettings.SignPackages}
assert not gpg_with_key.repository_sign_args
def test_repository_sign_args_skip_3(gpg: GPG) -> None:
"""
must return empty args if it is not set
"""
gpg.targets = {SignSettings.SignRepository}
assert not gpg.repository_sign_args
def test_repository_sign_args_skip_4(gpg: GPG) -> None:
"""
must return empty args if it is not set
"""
gpg.targets = {SignSettings.SignPackages, SignSettings.SignRepository}
assert not gpg.repository_sign_args
def test_sign_package_1(gpg_with_key: GPG, mocker: MockerFixture) -> None:
"""
must sign package
"""
result = [Path("a"), Path("a.sig")]
process_mock = mocker.patch("ahriman.core.sign.gpg.GPG.process", return_value=result)
gpg_with_key.targets = {SignSettings.SignPackages}
assert gpg_with_key.sign_package(Path("a"), "a") == result
process_mock.assert_called_once()
def test_sign_package_2(gpg_with_key: GPG, mocker: MockerFixture) -> None:
"""
must sign package
"""
result = [Path("a"), Path("a.sig")]
process_mock = mocker.patch("ahriman.core.sign.gpg.GPG.process", return_value=result)
gpg_with_key.targets = {SignSettings.SignPackages, SignSettings.SignRepository}
assert gpg_with_key.sign_package(Path("a"), "a") == result
process_mock.assert_called_once()
def test_sign_package_skip_1(gpg_with_key: GPG, mocker: MockerFixture) -> None:
"""
must not sign package if it is not set
"""
process_mock = mocker.patch("ahriman.core.sign.gpg.GPG.process")
gpg_with_key.targets = {}
gpg_with_key.sign_package(Path("a"), "a")
process_mock.assert_not_called()
def test_sign_package_skip_2(gpg_with_key: GPG, mocker: MockerFixture) -> None:
"""
must not sign package if it is not set
"""
process_mock = mocker.patch("ahriman.core.sign.gpg.GPG.process")
gpg_with_key.targets = {SignSettings.SignRepository}
gpg_with_key.sign_package(Path("a"), "a")
process_mock.assert_not_called()
def test_sign_package_skip_3(gpg: GPG, mocker: MockerFixture) -> None:
"""
must not sign package if it is not set
"""
process_mock = mocker.patch("ahriman.core.sign.gpg.GPG.process")
gpg.targets = {SignSettings.SignPackages}
gpg.sign_package(Path("a"), "a")
process_mock.assert_not_called()
def test_sign_package_skip_4(gpg: GPG, mocker: MockerFixture) -> None:
"""
must not sign package if it is not set
"""
process_mock = mocker.patch("ahriman.core.sign.gpg.GPG.process")
gpg.targets = {SignSettings.SignPackages, SignSettings.SignRepository}
gpg.sign_package(Path("a"), "a")
process_mock.assert_not_called()
def test_sign_repository_1(gpg_with_key: GPG, mocker: MockerFixture) -> None:
"""
must sign repository
"""
result = [Path("a"), Path("a.sig")]
process_mock = mocker.patch("ahriman.core.sign.gpg.GPG.process", return_value=result)
gpg_with_key.targets = {SignSettings.SignRepository}
assert gpg_with_key.sign_repository(Path("a")) == result
process_mock.assert_called_once()
def test_sign_repository_2(gpg_with_key: GPG, mocker: MockerFixture) -> None:
"""
must sign repository
"""
result = [Path("a"), Path("a.sig")]
process_mock = mocker.patch("ahriman.core.sign.gpg.GPG.process", return_value=result)
gpg_with_key.targets = {SignSettings.SignPackages, SignSettings.SignRepository}
assert gpg_with_key.sign_repository(Path("a")) == result
process_mock.assert_called_once()
def test_sign_repository_skip_1(gpg_with_key: GPG, mocker: MockerFixture) -> None:
"""
must not sign repository if it is not set
"""
process_mock = mocker.patch("ahriman.core.sign.gpg.GPG.process")
gpg_with_key.targets = {}
gpg_with_key.sign_repository(Path("a"))
process_mock.assert_not_called()
def test_sign_repository_skip_2(gpg_with_key: GPG, mocker: MockerFixture) -> None:
"""
must not sign repository if it is not set
"""
process_mock = mocker.patch("ahriman.core.sign.gpg.GPG.process")
gpg_with_key.targets = {SignSettings.SignPackages}
gpg_with_key.sign_repository(Path("a"))
process_mock.assert_not_called()
def test_sign_repository_skip_3(gpg: GPG, mocker: MockerFixture) -> None:
"""
must not sign repository if it is not set
"""
process_mock = mocker.patch("ahriman.core.sign.gpg.GPG.process")
gpg.targets = {SignSettings.SignRepository}
gpg.sign_repository(Path("a"))
process_mock.assert_not_called()
def test_sign_repository_skip_4(gpg: GPG, mocker: MockerFixture) -> None:
"""
must not sign repository if it is not set
"""
process_mock = mocker.patch("ahriman.core.sign.gpg.GPG.process")
gpg.targets = {SignSettings.SignPackages, SignSettings.SignRepository}
gpg.sign_repository(Path("a"))
process_mock.assert_not_called()

View File

@ -7,6 +7,22 @@ from ahriman.models.build_status import BuildStatusEnum
from ahriman.models.package import Package from ahriman.models.package import Package
def test_load_dummy_client(configuration: Configuration) -> None:
"""
must load dummy client if no settings set
"""
assert isinstance(Client.load(configuration), Client)
def test_load_full_client(configuration: Configuration) -> None:
"""
must load full client if no settings set
"""
configuration.set("web", "host", "localhost")
configuration.set("web", "port", "8080")
assert isinstance(Client.load(configuration), WebClient)
def test_add(client: Client, package_ahriman: Package) -> None: def test_add(client: Client, package_ahriman: Package) -> None:
""" """
must process package addition without errors must process package addition without errors
@ -98,19 +114,3 @@ def test_set_unknown(client: Client, package_ahriman: Package, mocker: MockerFix
client.set_unknown(package_ahriman) client.set_unknown(package_ahriman)
add_mock.assert_called_with(package_ahriman, BuildStatusEnum.Unknown) add_mock.assert_called_with(package_ahriman, BuildStatusEnum.Unknown)
def test_load_dummy_client(configuration: Configuration) -> None:
"""
must load dummy client if no settings set
"""
assert isinstance(Client.load("x86_64", configuration), Client)
def test_load_full_client(configuration: Configuration) -> None:
"""
must load full client if no settings set
"""
configuration.set("web", "host", "localhost")
configuration.set("web", "port", "8080")
assert isinstance(Client.load("x86_64", configuration), WebClient)

View File

@ -14,13 +14,20 @@ def test_from_path(mocker: MockerFixture) -> None:
load_logging_mock = mocker.patch("ahriman.core.configuration.Configuration.load_logging") load_logging_mock = mocker.patch("ahriman.core.configuration.Configuration.load_logging")
path = Path("path") path = Path("path")
config = Configuration.from_path(path, True) configuration = Configuration.from_path(path, "x86_64", True)
assert config.path == path assert configuration.path == path
read_mock.assert_called_with(path) read_mock.assert_called_with(path)
load_includes_mock.assert_called_once() load_includes_mock.assert_called_once()
load_logging_mock.assert_called_once() load_logging_mock.assert_called_once()
def test_section_name(configuration: Configuration) -> None:
"""
must return architecture specific group
"""
assert configuration.section_name("build", "x86_64") == "build:x86_64"
def test_absolute_path_for_absolute(configuration: Configuration) -> None: def test_absolute_path_for_absolute(configuration: Configuration) -> None:
""" """
must not change path for absolute path in settings must not change path for absolute path in settings
@ -46,20 +53,23 @@ def test_dump(configuration: Configuration) -> None:
""" """
dump must not be empty dump must not be empty
""" """
assert configuration.dump("x86_64") assert configuration.dump()
def test_dump_architecture_specific(configuration: Configuration) -> None: def test_dump_architecture_specific(configuration: Configuration) -> None:
""" """
dump must contain architecture specific settings dump must contain architecture specific settings
""" """
configuration.add_section("build_x86_64") section = configuration.section_name("build", "x86_64")
configuration.set("build_x86_64", "archbuild_flags", "") configuration.add_section(section)
configuration.set(section, "archbuild_flags", "hello flag")
configuration.merge_sections("x86_64")
dump = configuration.dump("x86_64") dump = configuration.dump()
assert dump assert dump
assert "build" not in dump assert "build" in dump
assert "build_x86_64" in dump assert section not in dump
assert dump["build"]["archbuild_flags"] == "hello flag"
def test_getlist(configuration: Configuration) -> None: def test_getlist(configuration: Configuration) -> None:
@ -87,23 +97,6 @@ def test_getlist_single(configuration: Configuration) -> None:
assert configuration.getlist("build", "test_list") == ["a"] assert configuration.getlist("build", "test_list") == ["a"]
def test_get_section_name(configuration: Configuration) -> None:
"""
must return architecture specific group
"""
configuration.add_section("build_x86_64")
configuration.set("build_x86_64", "archbuild_flags", "")
assert configuration.get_section_name("build", "x86_64") == "build_x86_64"
def test_get_section_name_missing(configuration: Configuration) -> None:
"""
must return default group if architecture depending group does not exist
"""
assert configuration.get_section_name("prefix", "suffix") == "prefix"
assert configuration.get_section_name("build", "x86_64") == "build"
def test_load_includes_missing(configuration: Configuration) -> None: def test_load_includes_missing(configuration: Configuration) -> None:
""" """
must not fail if not include directory found must not fail if not include directory found
@ -127,3 +120,16 @@ def test_load_logging_stderr(configuration: Configuration, mocker: MockerFixture
logging_mock = mocker.patch("logging.config.fileConfig") logging_mock = mocker.patch("logging.config.fileConfig")
configuration.load_logging(False) configuration.load_logging(False)
logging_mock.assert_not_called() logging_mock.assert_not_called()
def test_merge_sections_missing(configuration: Configuration) -> None:
"""
must merge create section if not exists
"""
section = configuration.section_name("build", "x86_64")
configuration.remove_section("build")
configuration.add_section(section)
configuration.set(section, "key", "value")
configuration.merge_sections("x86_64")
assert configuration.get("build", "key") == "value"

View File

@ -0,0 +1,16 @@
from pathlib import Path
from pytest_mock import MockerFixture
from ahriman.core.configuration import Configuration
from ahriman.core.upload.rsync import Rsync
def test_sync(configuration: Configuration, mocker: MockerFixture) -> None:
"""
must run sync command
"""
check_output_mock = mocker.patch("ahriman.core.upload.rsync.Rsync._check_output")
upload = Rsync("x86_64", configuration)
upload.sync(Path("path"))
check_output_mock.assert_called_once()

View File

@ -0,0 +1,16 @@
from pathlib import Path
from pytest_mock import MockerFixture
from ahriman.core.configuration import Configuration
from ahriman.core.upload.s3 import S3
def test_sync(configuration: Configuration, mocker: MockerFixture) -> None:
"""
must run sync command
"""
check_output_mock = mocker.patch("ahriman.core.upload.s3.S3._check_output")
upload = S3("x86_64", configuration)
upload.sync(Path("path"))
check_output_mock.assert_called_once()

View File

@ -0,0 +1,36 @@
import pytest
from pathlib import Path
from pytest_mock import MockerFixture
from ahriman.core.configuration import Configuration
from ahriman.core.exceptions import SyncFailed
from ahriman.core.upload.upload import Upload
from ahriman.models.upload_settings import UploadSettings
def test_upload_failure(configuration: Configuration, mocker: MockerFixture) -> None:
"""
must raise SyncFailed on errors
"""
mocker.patch("ahriman.core.upload.rsync.Rsync.sync", side_effect=Exception())
with pytest.raises(SyncFailed):
Upload.load("x86_64", configuration, UploadSettings.Rsync.name).run(Path("path"))
def test_upload_rsync(configuration: Configuration, mocker: MockerFixture) -> None:
"""
must upload via rsync
"""
upload_mock = mocker.patch("ahriman.core.upload.rsync.Rsync.sync")
Upload.load("x86_64", configuration, UploadSettings.Rsync.name).run(Path("path"))
upload_mock.assert_called_once()
def test_upload_s3(configuration: Configuration, mocker: MockerFixture) -> None:
"""
must upload via s3
"""
upload_mock = mocker.patch("ahriman.core.upload.s3.S3.sync")
Upload.load("x86_64", configuration, UploadSettings.S3.name).run(Path("path"))
upload_mock.assert_called_once()

View File

@ -1,8 +1,10 @@
import pytest import pytest
from unittest.mock import MagicMock, PropertyMock
from ahriman.models.build_status import BuildStatus, BuildStatusEnum from ahriman.models.build_status import BuildStatus, BuildStatusEnum
from ahriman.models.package import Package from ahriman.models.package import Package
from ahriman.models.package_desciption import PackageDescription from ahriman.models.package_description import PackageDescription
@pytest.fixture @pytest.fixture
@ -17,3 +19,33 @@ def package_tpacpi_bat_git() -> Package:
version="3.1.r12.g4959b52-1", version="3.1.r12.g4959b52-1",
aur_url="https://aur.archlinux.org", aur_url="https://aur.archlinux.org",
packages={"tpacpi-bat-git": PackageDescription()}) packages={"tpacpi-bat-git": PackageDescription()})
@pytest.fixture
def pyalpm_handle(pyalpm_package_ahriman: MagicMock) -> MagicMock:
mock = MagicMock()
mock.handle.load_pkg.return_value = pyalpm_package_ahriman
return mock
@pytest.fixture
def pyalpm_package_ahriman(package_ahriman: Package) -> MagicMock:
mock = MagicMock()
type(mock).base = PropertyMock(return_value=package_ahriman.base)
type(mock).name = PropertyMock(return_value=package_ahriman.base)
type(mock).version = PropertyMock(return_value=package_ahriman.version)
return mock
@pytest.fixture
def pyalpm_package_description_ahriman(package_description_ahriman: PackageDescription) -> MagicMock:
mock = MagicMock()
type(mock).arch = PropertyMock(return_value=package_description_ahriman.architecture)
type(mock).builddate = PropertyMock(return_value=package_description_ahriman.build_date)
type(mock).desc = PropertyMock(return_value=package_description_ahriman.description)
type(mock).groups = PropertyMock(return_value=package_description_ahriman.groups)
type(mock).isize = PropertyMock(return_value=package_description_ahriman.installed_size)
type(mock).licenses = PropertyMock(return_value=package_description_ahriman.licenses)
type(mock).size = PropertyMock(return_value=package_description_ahriman.archive_size)
type(mock).url = PropertyMock(return_value=package_description_ahriman.url)
return mock

View File

@ -1,6 +1,10 @@
import pytest
from pathlib import Path from pathlib import Path
from pytest_mock import MockerFixture from pytest_mock import MockerFixture
from unittest.mock import MagicMock, PropertyMock
from ahriman.core.exceptions import InvalidPackageInfo
from ahriman.models.package import Package from ahriman.models.package import Package
from ahriman.models.repository_paths import RepositoryPaths from ahriman.models.repository_paths import RepositoryPaths
@ -14,6 +18,17 @@ def test_git_url(package_ahriman: Package) -> None:
assert package_ahriman.base in package_ahriman.git_url assert package_ahriman.base in package_ahriman.git_url
def test_groups(package_ahriman: Package) -> None:
"""
must return list of groups for each package
"""
assert all(
all(group in package_ahriman.groups for group in package.groups)
for package in package_ahriman.packages.values()
)
assert sorted(package_ahriman.groups) == package_ahriman.groups
def test_is_single_package_false(package_python_schedule: Package) -> None: def test_is_single_package_false(package_python_schedule: Package) -> None:
""" """
python-schedule must not be single package python-schedule must not be single package
@ -42,6 +57,17 @@ def test_is_vcs_true(package_tpacpi_bat_git: Package) -> None:
assert package_tpacpi_bat_git.is_vcs assert package_tpacpi_bat_git.is_vcs
def test_licenses(package_ahriman: Package) -> None:
"""
must return list of licenses for each package
"""
assert all(
all(lic in package_ahriman.licenses for lic in package.licenses)
for package in package_ahriman.packages.values()
)
assert sorted(package_ahriman.licenses) == package_ahriman.licenses
def test_web_url(package_ahriman: Package) -> None: def test_web_url(package_ahriman: Package) -> None:
""" """
must generate valid web url must generate valid web url
@ -50,6 +76,44 @@ def test_web_url(package_ahriman: Package) -> None:
assert package_ahriman.base in package_ahriman.web_url assert package_ahriman.base in package_ahriman.web_url
def test_from_archive(package_ahriman: Package, pyalpm_handle: MagicMock, mocker: MockerFixture) -> None:
"""
must construct package from alpm library
"""
mocker.patch("ahriman.models.package_description.PackageDescription.from_package",
return_value=package_ahriman.packages[package_ahriman.base])
assert Package.from_archive(Path("path"), pyalpm_handle, package_ahriman.aur_url) == package_ahriman
def test_from_aur(package_ahriman: Package, mocker: MockerFixture) -> None:
"""
must construct package from aur
"""
mock = MagicMock()
type(mock).name = PropertyMock(return_value=package_ahriman.base)
type(mock).package_base = PropertyMock(return_value=package_ahriman.base)
type(mock).version = PropertyMock(return_value=package_ahriman.version)
mocker.patch("aur.info", return_value=mock)
package = Package.from_aur(package_ahriman.base, package_ahriman.aur_url)
assert package_ahriman.base == package.base
assert package_ahriman.version == package.version
assert package_ahriman.packages.keys() == package.packages.keys()
def test_from_build(package_ahriman: Package, mocker: MockerFixture, resource_path_root: Path) -> None:
"""
must construct package from srcinfo
"""
srcinfo = (resource_path_root / "models" / "package_ahriman_srcinfo").read_text()
mocker.patch("pathlib.Path.read_text", return_value=srcinfo)
package = Package.from_build(Path("path"), package_ahriman.aur_url)
assert package_ahriman.packages.keys() == package.packages.keys()
package_ahriman.packages = package.packages # we are not going to test PackageDescription here
assert package_ahriman == package
def test_from_json_view_1(package_ahriman: Package) -> None: def test_from_json_view_1(package_ahriman: Package) -> None:
""" """
must construct same object from json must construct same object from json
@ -71,17 +135,69 @@ def test_from_json_view_3(package_tpacpi_bat_git: Package) -> None:
assert Package.from_json(package_tpacpi_bat_git.view()) == package_tpacpi_bat_git assert Package.from_json(package_tpacpi_bat_git.view()) == package_tpacpi_bat_git
def test_load_from_archive(package_ahriman: Package, pyalpm_handle: MagicMock, mocker: MockerFixture) -> None:
"""
must load package from package archive
"""
mocker.patch("pathlib.Path.is_file", return_value=True)
load_mock = mocker.patch("ahriman.models.package.Package.from_archive")
Package.load(Path("path"), pyalpm_handle, package_ahriman.aur_url)
load_mock.assert_called_once()
def test_load_from_aur(package_ahriman: Package, pyalpm_handle: MagicMock, mocker: MockerFixture) -> None:
"""
must load package from AUR
"""
load_mock = mocker.patch("ahriman.models.package.Package.from_aur")
Package.load(Path("path"), pyalpm_handle, package_ahriman.aur_url)
load_mock.assert_called_once()
def test_load_from_build(package_ahriman: Package, pyalpm_handle: MagicMock, mocker: MockerFixture) -> None:
"""
must load package from build directory
"""
mocker.patch("pathlib.Path.is_dir", return_value=True)
load_mock = mocker.patch("ahriman.models.package.Package.from_build")
Package.load(Path("path"), pyalpm_handle, package_ahriman.aur_url)
load_mock.assert_called_once()
def test_load_failure(package_ahriman: Package, pyalpm_handle: MagicMock, mocker: MockerFixture) -> None:
"""
must raise InvalidPackageInfo on exception
"""
mocker.patch("pathlib.Path.is_dir", side_effect=InvalidPackageInfo("exception!"))
with pytest.raises(InvalidPackageInfo):
Package.load(Path("path"), pyalpm_handle, package_ahriman.aur_url)
mocker.patch("pathlib.Path.is_dir", side_effect=Exception())
with pytest.raises(InvalidPackageInfo):
Package.load(Path("path"), pyalpm_handle, package_ahriman.aur_url)
def test_dependencies_with_version(mocker: MockerFixture, resource_path_root: Path) -> None: def test_dependencies_with_version(mocker: MockerFixture, resource_path_root: Path) -> None:
""" """
must load correct list of dependencies with version must load correct list of dependencies with version
""" """
srcinfo = (resource_path_root / "models" / "package_yay_srcinfo").read_text() srcinfo = (resource_path_root / "models" / "package_yay_srcinfo").read_text()
mocker.patch("pathlib.Path.read_text", return_value=srcinfo) mocker.patch("pathlib.Path.read_text", return_value=srcinfo)
assert Package.dependencies(Path("path")) == {"git", "go", "pacman"} assert Package.dependencies(Path("path")) == {"git", "go", "pacman"}
def test_full_version() -> None:
"""
must construct full version
"""
assert Package.full_version("1", "r2388.d30e3201", "1") == "1:r2388.d30e3201-1"
assert Package.full_version(None, "0.12.1", "1") == "0.12.1-1"
def test_actual_version(package_ahriman: Package, repository_paths: RepositoryPaths) -> None: def test_actual_version(package_ahriman: Package, repository_paths: RepositoryPaths) -> None:
""" """
must return same actual_version as version is must return same actual_version as version is
@ -95,7 +211,6 @@ def test_actual_version_vcs(package_tpacpi_bat_git: Package, repository_paths: R
must return valid actual_version for VCS package must return valid actual_version for VCS package
""" """
srcinfo = (resource_path_root / "models" / "package_tpacpi-bat-git_srcinfo").read_text() srcinfo = (resource_path_root / "models" / "package_tpacpi-bat-git_srcinfo").read_text()
mocker.patch("ahriman.models.package.Package._check_output", return_value=srcinfo) mocker.patch("ahriman.models.package.Package._check_output", return_value=srcinfo)
mocker.patch("ahriman.core.build_tools.task.Task.fetch") mocker.patch("ahriman.core.build_tools.task.Task.fetch")

View File

@ -1,4 +1,6 @@
from ahriman.models.package_desciption import PackageDescription from unittest.mock import MagicMock
from ahriman.models.package_description import PackageDescription
def test_filepath(package_description_ahriman: PackageDescription) -> None: def test_filepath(package_description_ahriman: PackageDescription) -> None:
@ -15,3 +17,13 @@ def test_filepath_empty(package_description_ahriman: PackageDescription) -> None
""" """
package_description_ahriman.filename = None package_description_ahriman.filename = None
assert package_description_ahriman.filepath is None assert package_description_ahriman.filepath is None
def test_from_package(package_description_ahriman: PackageDescription,
pyalpm_package_description_ahriman: MagicMock) -> None:
"""
must construct description from package object
"""
package_description = PackageDescription.from_package(pyalpm_package_description_ahriman,
package_description_ahriman.filepath)
assert package_description_ahriman == package_description

View File

@ -36,10 +36,10 @@ def test_run(application: web.Application, mocker: MockerFixture) -> None:
""" """
host = "localhost" host = "localhost"
port = 8080 port = 8080
application["config"].set("web", "host", host) application["configuration"].set("web", "host", host)
application["config"].set("web", "port", str(port)) application["configuration"].set("web", "port", str(port))
run_app_mock = mocker.patch("aiohttp.web.run_app") run_application_mock = mocker.patch("aiohttp.web.run_app")
run_server(application) run_server(application)
run_app_mock.assert_called_with(application, host=host, port=port, run_application_mock.assert_called_with(application, host=host, port=port,
handle_signals=False, access_log=pytest.helpers.anyvar(int)) handle_signals=False, access_log=pytest.helpers.anyvar(int))

View File

@ -21,7 +21,6 @@ root = /var/lib/ahriman
[sign] [sign]
target = target =
key =
[report] [report]
target = target =
@ -36,10 +35,12 @@ template_path = ../web/templates/repo-index.jinja2
target = target =
[rsync] [rsync]
command = rsync --archive --verbose --compress --partial --delete
remote = remote =
[s3] [s3]
bucket = bucket =
command = aws s3 sync --quiet --delete
[web] [web]
templates = ../web/templates templates = ../web/templates

View File

@ -0,0 +1,34 @@
pkgbase = ahriman
pkgdesc = ArcHlinux ReposItory MANager
pkgver = 0.12.1
pkgrel = 1
url = https://github.com/arcan1s/ahriman
arch = any
license = GPL3
makedepends = python-pip
depends = devtools
depends = git
depends = pyalpm
depends = python-aur
depends = python-srcinfo
optdepends = aws-cli: sync to s3
optdepends = breezy: -bzr packages support
optdepends = darcs: -darcs packages support
optdepends = gnupg: package and repository sign
optdepends = mercurial: -hg packages support
optdepends = python-aiohttp: web server
optdepends = python-aiohttp-jinja2: web server
optdepends = python-jinja: html report generation
optdepends = python-requests: web server
optdepends = rsync: sync by using rsync
optdepends = subversion: -svn packages support
backup = etc/ahriman.ini
backup = etc/ahriman.ini.d/logging.ini
source = https://github.com/arcan1s/ahriman/releases/download/0.12.1/ahriman-0.12.1-src.tar.xz
source = ahriman.sysusers
source = ahriman.tmpfiles
sha512sums = 8acc57f937d587ca665c29092cadddbaf3ba0b80e870b80d1551e283aba8f21306f9030a26fec8c71ab5863316f5f5f061b7ddc63cdff9e6d5a885f28ef1893d
sha512sums = 13718afec2c6786a18f0b223ef8e58dccf0688bca4cdbe203f14071f5031ed20120eb0ce38b52c76cfd6e8b6581a9c9eaa2743eb11abbaca637451a84c33f075
sha512sums = 55b20f6da3d66e7bbf2add5d95a3b60632df121717d25a993e56e737d14f51fe063eb6f1b38bd81cc32e05db01c0c1d80aaa720c45cde87f238d8b46cdb8cbc4
pkgname = ahriman