Compare commits

...

13 Commits

Author SHA1 Message Date
ec0550a275 Release 2.7.1 2023-03-06 01:15:47 +02:00
df23be9269 gracefully terminate web server
In previous revisions server was terminated by itself, thus no lock or
socket was removed. In new version, graceful termination of the queue
has been added as well as server now handles singals
2023-03-06 01:13:41 +02:00
a8c40a6b87 replace InitializeException with InitializeError in docs 2023-03-02 11:07:59 +02:00
a274f91677 simplify login ttl processing 2023-02-24 16:52:55 +02:00
13faf66bdb add more validation rules 2023-02-23 15:18:56 +02:00
4fb9335df9 add ability to read cookie secret from config 2023-02-22 18:47:56 +02:00
d517d8bfbb Release 2.7.0 2023-02-20 03:05:08 +02:00
37e57c13c8 update dependencies before build (#91)
Old implementation has used add step in order to fetch dependencies,
which could lead to build errors in case if dependency list was updated.

New solution uses dependencies which are declared at current version and
fetch them (if required and if enabled) before update process.

Closes #90
2023-02-12 06:02:30 +03:00
19bb19e9f5 handle .gitignore file correctly in remote push trigger 2023-02-11 04:41:24 +02:00
3a4e8f4d97 mask mypy warning
The newest mypy produces the following warning:

src/ahriman/application/handlers/search.py:43: error: Non-overlapping identity check (left operand type: "Union[_DefaultFactory[Any], Literal[_MISSING_TYPE.MISSING]]", right operand type: "Type[List[Any]]")  [comparison-overlap]

which is more likely caused by updated dataclass models to protoocol (however decorators are still calllable). This commit masks problematic line from checking
2023-02-09 22:46:08 +02:00
4db8ad8e8d hide passwords and secrets from repo-config subcommand by default 2023-02-05 16:44:48 +02:00
117f096d41 note about local database update (see #85) 2023-01-31 14:47:58 +02:00
917ec48be5 handle architecture specific fields for dependencies
This change requires srcinfo at least 0.1.2 version. Unfortunatelly aur
api don't support architecture specific arrays for now, so we just leave
it as is

Closes #82
2023-01-31 14:34:09 +02:00
66 changed files with 4198 additions and 4232 deletions

View File

@ -1 +1 @@
skips: ['B101', 'B105', 'B106', 'B404']
skips: ['B101', 'B104', 'B105', 'B106', 'B404']

View File

@ -533,5 +533,5 @@ valid-metaclass-classmethod-first-arg=cls
# Exceptions that will emit a warning when being caught. Defaults to
# "BaseException, Exception".
overgeneral-exceptions=BaseException,
Exception
overgeneral-exceptions=builtins.BaseException,
builtins.Exception

File diff suppressed because it is too large Load Diff

Before

Width:  |  Height:  |  Size: 673 KiB

After

Width:  |  Height:  |  Size: 660 KiB

View File

@ -1,4 +1,4 @@
.TH AHRIMAN "1" "2023\-01\-27" "ahriman" "Generated Python Manual"
.TH AHRIMAN "1" "2023\-03\-06" "ahriman" "Generated Python Manual"
.SH NAME
ahriman
.SH SYNOPSIS
@ -215,8 +215,8 @@ usage: ahriman help\-version [\-h]
print application and its dependencies versions
.SH COMMAND \fI\,'ahriman package\-add'\/\fR
usage: ahriman package\-add [\-h] [\-e] [\-n] [\-y] [\-s {auto,archive,aur,directory,local,remote,repository}]
[\-\-without\-dependencies]
usage: ahriman package\-add [\-h] [\-\-dependencies | \-\-no\-dependencies] [\-e] [\-n] [\-y]
[\-s {auto,archive,aur,directory,local,remote,repository}]
package [package ...]
add existing or new package to the build queue
@ -226,6 +226,10 @@ add existing or new package to the build queue
package source (base name, path to local files, remote URL)
.SH OPTIONS \fI\,'ahriman package\-add'\/\fR
.TP
\fB\-\-dependencies\fR, \fB\-\-no\-dependencies\fR
process missing package dependencies (default: True)
.TP
\fB\-e\fR, \fB\-\-exit\-code\fR
return non\-zero exit status if result is empty
@ -242,10 +246,6 @@ download fresh package databases from the mirror before actions, \-yy to force r
\fB\-s\fR \fI\,{auto,archive,aur,directory,local,remote,repository}\/\fR, \fB\-\-source\fR \fI\,{auto,archive,aur,directory,local,remote,repository}\/\fR
explicitly specify the package source for this command
.TP
\fB\-\-without\-dependencies\fR
do not add dependencies
.SH COMMAND \fI\,'ahriman package\-remove'\/\fR
usage: ahriman package\-remove [\-h] package [package ...]
@ -401,8 +401,8 @@ fetch actual version of VCS packages (default: True)
download fresh package databases from the mirror before actions, \-yy to force refresh even if up to date
.SH COMMAND \fI\,'ahriman repo\-daemon'\/\fR
usage: ahriman repo\-daemon [\-h] [\-i INTERVAL] [\-\-aur | \-\-no\-aur] [\-\-local | \-\-no\-local] [\-\-manual | \-\-no\-manual]
[\-\-vcs | \-\-no\-vcs] [\-y]
usage: ahriman repo\-daemon [\-h] [\-i INTERVAL] [\-\-aur | \-\-no\-aur] [\-\-dependencies | \-\-no\-dependencies]
[\-\-local | \-\-no\-local] [\-\-manual | \-\-no\-manual] [\-\-vcs | \-\-no\-vcs] [\-y]
start process which periodically will run update process
@ -415,6 +415,10 @@ interval between runs in seconds
\fB\-\-aur\fR, \fB\-\-no\-aur\fR
enable or disable checking for AUR updates (default: True)
.TP
\fB\-\-dependencies\fR, \fB\-\-no\-dependencies\fR
process missing package dependencies (default: True)
.TP
\fB\-\-local\fR, \fB\-\-no\-local\fR
enable or disable checking of local packages for updates (default: True)
@ -523,8 +527,8 @@ run triggers on empty build result as configured by settings
instead of running all triggers as set by configuration, just process specified ones in order of mention
.SH COMMAND \fI\,'ahriman repo\-update'\/\fR
usage: ahriman repo\-update [\-h] [\-\-dry\-run] [\-e] [\-\-aur | \-\-no\-aur] [\-\-local | \-\-no\-local] [\-\-manual | \-\-no\-manual]
[\-\-vcs | \-\-no\-vcs] [\-y]
usage: ahriman repo\-update [\-h] [\-\-aur | \-\-no\-aur] [\-\-dependencies | \-\-no\-dependencies] [\-\-dry\-run] [\-e]
[\-\-local | \-\-no\-local] [\-\-manual | \-\-no\-manual] [\-\-vcs | \-\-no\-vcs] [\-y]
[package ...]
check for packages updates and run build process if requested
@ -534,6 +538,14 @@ check for packages updates and run build process if requested
filter check by package base
.SH OPTIONS \fI\,'ahriman repo\-update'\/\fR
.TP
\fB\-\-aur\fR, \fB\-\-no\-aur\fR
enable or disable checking for AUR updates (default: True)
.TP
\fB\-\-dependencies\fR, \fB\-\-no\-dependencies\fR
process missing package dependencies (default: True)
.TP
\fB\-\-dry\-run\fR
just perform check for updates, same as check command
@ -542,10 +554,6 @@ just perform check for updates, same as check command
\fB\-e\fR, \fB\-\-exit\-code\fR
return non\-zero exit status if result is empty
.TP
\fB\-\-aur\fR, \fB\-\-no\-aur\fR
enable or disable checking for AUR updates (default: True)
.TP
\fB\-\-local\fR, \fB\-\-no\-local\fR
enable or disable checking of local packages for updates (default: True)
@ -590,10 +598,15 @@ clear directory with built packages (default: False)
clear directory with pacman local database cache (default: False)
.SH COMMAND \fI\,'ahriman service\-config'\/\fR
usage: ahriman service\-config [\-h]
usage: ahriman service\-config [\-h] [\-\-secure | \-\-no\-secure]
dump configuration for the specified architecture
.SH OPTIONS \fI\,'ahriman service\-config'\/\fR
.TP
\fB\-\-secure\fR, \fB\-\-no\-secure\fR
hide passwords and secrets from output (default: True)
.SH COMMAND \fI\,'ahriman service\-config\-validate'\/\fR
usage: ahriman service\-config\-validate [\-h] [\-e]

View File

@ -52,6 +52,14 @@ ahriman.core.database.migrations.m005\_make\_opt\_depends module
:no-undoc-members:
:show-inheritance:
ahriman.core.database.migrations.m006\_packages\_architecture\_required module
------------------------------------------------------------------------------
.. automodule:: ahriman.core.database.migrations.m006_packages_architecture_required
:members:
:no-undoc-members:
:show-inheritance:
Module contents
---------------

View File

@ -196,14 +196,6 @@ ahriman.models.user\_access module
:no-undoc-members:
:show-inheritance:
ahriman.models.user\_identity module
------------------------------------
.. automodule:: ahriman.models.user_identity
:members:
:no-undoc-members:
:show-inheritance:
Module contents
---------------

View File

@ -10,9 +10,9 @@ _shtab_ahriman_help_commands_unsafe_option_strings=('-h' '--help' '--command')
_shtab_ahriman_help_updates_option_strings=('-h' '--help' '-e' '--exit-code')
_shtab_ahriman_help_version_option_strings=('-h' '--help')
_shtab_ahriman_version_option_strings=('-h' '--help')
_shtab_ahriman_package_add_option_strings=('-h' '--help' '-e' '--exit-code' '-n' '--now' '-y' '--refresh' '-s' '--source' '--without-dependencies')
_shtab_ahriman_add_option_strings=('-h' '--help' '-e' '--exit-code' '-n' '--now' '-y' '--refresh' '-s' '--source' '--without-dependencies')
_shtab_ahriman_package_update_option_strings=('-h' '--help' '-e' '--exit-code' '-n' '--now' '-y' '--refresh' '-s' '--source' '--without-dependencies')
_shtab_ahriman_package_add_option_strings=('-h' '--help' '--dependencies' '--no-dependencies' '-e' '--exit-code' '-n' '--now' '-y' '--refresh' '-s' '--source')
_shtab_ahriman_add_option_strings=('-h' '--help' '--dependencies' '--no-dependencies' '-e' '--exit-code' '-n' '--now' '-y' '--refresh' '-s' '--source')
_shtab_ahriman_package_update_option_strings=('-h' '--help' '--dependencies' '--no-dependencies' '-e' '--exit-code' '-n' '--now' '-y' '--refresh' '-s' '--source')
_shtab_ahriman_package_remove_option_strings=('-h' '--help')
_shtab_ahriman_remove_option_strings=('-h' '--help')
_shtab_ahriman_package_status_option_strings=('-h' '--help' '--ahriman' '-e' '--exit-code' '--info' '--no-info' '-s' '--status')
@ -27,8 +27,8 @@ _shtab_ahriman_patch_set_add_option_strings=('-h' '--help' '-t' '--track')
_shtab_ahriman_repo_backup_option_strings=('-h' '--help')
_shtab_ahriman_repo_check_option_strings=('-h' '--help' '-e' '--exit-code' '--vcs' '--no-vcs' '-y' '--refresh')
_shtab_ahriman_check_option_strings=('-h' '--help' '-e' '--exit-code' '--vcs' '--no-vcs' '-y' '--refresh')
_shtab_ahriman_repo_daemon_option_strings=('-h' '--help' '-i' '--interval' '--aur' '--no-aur' '--local' '--no-local' '--manual' '--no-manual' '--vcs' '--no-vcs' '-y' '--refresh')
_shtab_ahriman_daemon_option_strings=('-h' '--help' '-i' '--interval' '--aur' '--no-aur' '--local' '--no-local' '--manual' '--no-manual' '--vcs' '--no-vcs' '-y' '--refresh')
_shtab_ahriman_repo_daemon_option_strings=('-h' '--help' '-i' '--interval' '--aur' '--no-aur' '--dependencies' '--no-dependencies' '--local' '--no-local' '--manual' '--no-manual' '--vcs' '--no-vcs' '-y' '--refresh')
_shtab_ahriman_daemon_option_strings=('-h' '--help' '-i' '--interval' '--aur' '--no-aur' '--dependencies' '--no-dependencies' '--local' '--no-local' '--manual' '--no-manual' '--vcs' '--no-vcs' '-y' '--refresh')
_shtab_ahriman_repo_rebuild_option_strings=('-h' '--help' '--depends-on' '--dry-run' '--from-database' '-e' '--exit-code')
_shtab_ahriman_rebuild_option_strings=('-h' '--help' '--depends-on' '--dry-run' '--from-database' '-e' '--exit-code')
_shtab_ahriman_repo_remove_unknown_option_strings=('-h' '--help' '--dry-run')
@ -43,14 +43,14 @@ _shtab_ahriman_repo_sync_option_strings=('-h' '--help')
_shtab_ahriman_sync_option_strings=('-h' '--help')
_shtab_ahriman_repo_tree_option_strings=('-h' '--help')
_shtab_ahriman_repo_triggers_option_strings=('-h' '--help')
_shtab_ahriman_repo_update_option_strings=('-h' '--help' '--dry-run' '-e' '--exit-code' '--aur' '--no-aur' '--local' '--no-local' '--manual' '--no-manual' '--vcs' '--no-vcs' '-y' '--refresh')
_shtab_ahriman_update_option_strings=('-h' '--help' '--dry-run' '-e' '--exit-code' '--aur' '--no-aur' '--local' '--no-local' '--manual' '--no-manual' '--vcs' '--no-vcs' '-y' '--refresh')
_shtab_ahriman_repo_update_option_strings=('-h' '--help' '--aur' '--no-aur' '--dependencies' '--no-dependencies' '--dry-run' '-e' '--exit-code' '--local' '--no-local' '--manual' '--no-manual' '--vcs' '--no-vcs' '-y' '--refresh')
_shtab_ahriman_update_option_strings=('-h' '--help' '--aur' '--no-aur' '--dependencies' '--no-dependencies' '--dry-run' '-e' '--exit-code' '--local' '--no-local' '--manual' '--no-manual' '--vcs' '--no-vcs' '-y' '--refresh')
_shtab_ahriman_service_clean_option_strings=('-h' '--help' '--cache' '--no-cache' '--chroot' '--no-chroot' '--manual' '--no-manual' '--packages' '--no-packages' '--pacman' '--no-pacman')
_shtab_ahriman_clean_option_strings=('-h' '--help' '--cache' '--no-cache' '--chroot' '--no-chroot' '--manual' '--no-manual' '--packages' '--no-packages' '--pacman' '--no-pacman')
_shtab_ahriman_repo_clean_option_strings=('-h' '--help' '--cache' '--no-cache' '--chroot' '--no-chroot' '--manual' '--no-manual' '--packages' '--no-packages' '--pacman' '--no-pacman')
_shtab_ahriman_service_config_option_strings=('-h' '--help')
_shtab_ahriman_config_option_strings=('-h' '--help')
_shtab_ahriman_repo_config_option_strings=('-h' '--help')
_shtab_ahriman_service_config_option_strings=('-h' '--help' '--secure' '--no-secure')
_shtab_ahriman_config_option_strings=('-h' '--help' '--secure' '--no-secure')
_shtab_ahriman_repo_config_option_strings=('-h' '--help' '--secure' '--no-secure')
_shtab_ahriman_service_config_validate_option_strings=('-h' '--help' '-e' '--exit-code')
_shtab_ahriman_config_validate_option_strings=('-h' '--help' '-e' '--exit-code')
_shtab_ahriman_repo_config_validate_option_strings=('-h' '--help' '-e' '--exit-code')
@ -139,33 +139,36 @@ _shtab_ahriman_version___help_nargs=0
_shtab_ahriman_package_add_pos_0_nargs=+
_shtab_ahriman_package_add__h_nargs=0
_shtab_ahriman_package_add___help_nargs=0
_shtab_ahriman_package_add___dependencies_nargs=0
_shtab_ahriman_package_add___no_dependencies_nargs=0
_shtab_ahriman_package_add__e_nargs=0
_shtab_ahriman_package_add___exit_code_nargs=0
_shtab_ahriman_package_add__n_nargs=0
_shtab_ahriman_package_add___now_nargs=0
_shtab_ahriman_package_add__y_nargs=0
_shtab_ahriman_package_add___refresh_nargs=0
_shtab_ahriman_package_add___without_dependencies_nargs=0
_shtab_ahriman_add_pos_0_nargs=+
_shtab_ahriman_add__h_nargs=0
_shtab_ahriman_add___help_nargs=0
_shtab_ahriman_add___dependencies_nargs=0
_shtab_ahriman_add___no_dependencies_nargs=0
_shtab_ahriman_add__e_nargs=0
_shtab_ahriman_add___exit_code_nargs=0
_shtab_ahriman_add__n_nargs=0
_shtab_ahriman_add___now_nargs=0
_shtab_ahriman_add__y_nargs=0
_shtab_ahriman_add___refresh_nargs=0
_shtab_ahriman_add___without_dependencies_nargs=0
_shtab_ahriman_package_update_pos_0_nargs=+
_shtab_ahriman_package_update__h_nargs=0
_shtab_ahriman_package_update___help_nargs=0
_shtab_ahriman_package_update___dependencies_nargs=0
_shtab_ahriman_package_update___no_dependencies_nargs=0
_shtab_ahriman_package_update__e_nargs=0
_shtab_ahriman_package_update___exit_code_nargs=0
_shtab_ahriman_package_update__n_nargs=0
_shtab_ahriman_package_update___now_nargs=0
_shtab_ahriman_package_update__y_nargs=0
_shtab_ahriman_package_update___refresh_nargs=0
_shtab_ahriman_package_update___without_dependencies_nargs=0
_shtab_ahriman_package_remove_pos_0_nargs=+
_shtab_ahriman_package_remove__h_nargs=0
_shtab_ahriman_package_remove___help_nargs=0
@ -231,6 +234,8 @@ _shtab_ahriman_repo_daemon__h_nargs=0
_shtab_ahriman_repo_daemon___help_nargs=0
_shtab_ahriman_repo_daemon___aur_nargs=0
_shtab_ahriman_repo_daemon___no_aur_nargs=0
_shtab_ahriman_repo_daemon___dependencies_nargs=0
_shtab_ahriman_repo_daemon___no_dependencies_nargs=0
_shtab_ahriman_repo_daemon___local_nargs=0
_shtab_ahriman_repo_daemon___no_local_nargs=0
_shtab_ahriman_repo_daemon___manual_nargs=0
@ -243,6 +248,8 @@ _shtab_ahriman_daemon__h_nargs=0
_shtab_ahriman_daemon___help_nargs=0
_shtab_ahriman_daemon___aur_nargs=0
_shtab_ahriman_daemon___no_aur_nargs=0
_shtab_ahriman_daemon___dependencies_nargs=0
_shtab_ahriman_daemon___no_dependencies_nargs=0
_shtab_ahriman_daemon___local_nargs=0
_shtab_ahriman_daemon___no_local_nargs=0
_shtab_ahriman_daemon___manual_nargs=0
@ -295,11 +302,13 @@ _shtab_ahriman_repo_triggers___help_nargs=0
_shtab_ahriman_repo_update_pos_0_nargs=*
_shtab_ahriman_repo_update__h_nargs=0
_shtab_ahriman_repo_update___help_nargs=0
_shtab_ahriman_repo_update___aur_nargs=0
_shtab_ahriman_repo_update___no_aur_nargs=0
_shtab_ahriman_repo_update___dependencies_nargs=0
_shtab_ahriman_repo_update___no_dependencies_nargs=0
_shtab_ahriman_repo_update___dry_run_nargs=0
_shtab_ahriman_repo_update__e_nargs=0
_shtab_ahriman_repo_update___exit_code_nargs=0
_shtab_ahriman_repo_update___aur_nargs=0
_shtab_ahriman_repo_update___no_aur_nargs=0
_shtab_ahriman_repo_update___local_nargs=0
_shtab_ahriman_repo_update___no_local_nargs=0
_shtab_ahriman_repo_update___manual_nargs=0
@ -311,11 +320,13 @@ _shtab_ahriman_repo_update___refresh_nargs=0
_shtab_ahriman_update_pos_0_nargs=*
_shtab_ahriman_update__h_nargs=0
_shtab_ahriman_update___help_nargs=0
_shtab_ahriman_update___aur_nargs=0
_shtab_ahriman_update___no_aur_nargs=0
_shtab_ahriman_update___dependencies_nargs=0
_shtab_ahriman_update___no_dependencies_nargs=0
_shtab_ahriman_update___dry_run_nargs=0
_shtab_ahriman_update__e_nargs=0
_shtab_ahriman_update___exit_code_nargs=0
_shtab_ahriman_update___aur_nargs=0
_shtab_ahriman_update___no_aur_nargs=0
_shtab_ahriman_update___local_nargs=0
_shtab_ahriman_update___no_local_nargs=0
_shtab_ahriman_update___manual_nargs=0
@ -362,10 +373,16 @@ _shtab_ahriman_repo_clean___pacman_nargs=0
_shtab_ahriman_repo_clean___no_pacman_nargs=0
_shtab_ahriman_service_config__h_nargs=0
_shtab_ahriman_service_config___help_nargs=0
_shtab_ahriman_service_config___secure_nargs=0
_shtab_ahriman_service_config___no_secure_nargs=0
_shtab_ahriman_config__h_nargs=0
_shtab_ahriman_config___help_nargs=0
_shtab_ahriman_config___secure_nargs=0
_shtab_ahriman_config___no_secure_nargs=0
_shtab_ahriman_repo_config__h_nargs=0
_shtab_ahriman_repo_config___help_nargs=0
_shtab_ahriman_repo_config___secure_nargs=0
_shtab_ahriman_repo_config___no_secure_nargs=0
_shtab_ahriman_service_config_validate__h_nargs=0
_shtab_ahriman_service_config_validate___help_nargs=0
_shtab_ahriman_service_config_validate__e_nargs=0

View File

@ -87,11 +87,11 @@ _shtab_ahriman_options=(
_shtab_ahriman_add_options=(
"(- : *)"{-h,--help}"[show this help message and exit]"
{--dependencies,--no-dependencies}"[process missing package dependencies (default\: \%(default)s)]:dependencies:"
{-e,--exit-code}"[return non-zero exit status if result is empty]"
{-n,--now}"[run update function after]"
"*"{-y,--refresh}"[download fresh package databases from the mirror before actions, -yy to force refresh even if up to date]"
{-s,--source}"[explicitly specify the package source for this command]:source:(auto archive aur directory local remote repository)"
"--without-dependencies[do not add dependencies]"
"(*):package source (base name, path to local files, remote URL):"
)
@ -122,6 +122,7 @@ _shtab_ahriman_clean_options=(
_shtab_ahriman_config_options=(
"(- : *)"{-h,--help}"[show this help message and exit]"
{--secure,--no-secure}"[hide passwords and secrets from output (default\: \%(default)s)]:secure:"
)
_shtab_ahriman_config_validate_options=(
@ -133,6 +134,7 @@ _shtab_ahriman_daemon_options=(
"(- : *)"{-h,--help}"[show this help message and exit]"
{-i,--interval}"[interval between runs in seconds]:interval:"
{--aur,--no-aur}"[enable or disable checking for AUR updates (default\: \%(default)s)]:aur:"
{--dependencies,--no-dependencies}"[process missing package dependencies (default\: \%(default)s)]:dependencies:"
{--local,--no-local}"[enable or disable checking of local packages for updates (default\: \%(default)s)]:local:"
{--manual,--no-manual}"[include or exclude manual updates (default\: \%(default)s)]:manual:"
{--vcs,--no-vcs}"[fetch actual version of VCS packages (default\: \%(default)s)]:vcs:"
@ -182,11 +184,11 @@ _shtab_ahriman_key_import_options=(
_shtab_ahriman_package_add_options=(
"(- : *)"{-h,--help}"[show this help message and exit]"
{--dependencies,--no-dependencies}"[process missing package dependencies (default\: \%(default)s)]:dependencies:"
{-e,--exit-code}"[return non-zero exit status if result is empty]"
{-n,--now}"[run update function after]"
"*"{-y,--refresh}"[download fresh package databases from the mirror before actions, -yy to force refresh even if up to date]"
{-s,--source}"[explicitly specify the package source for this command]:source:(auto archive aur directory local remote repository)"
"--without-dependencies[do not add dependencies]"
"(*):package source (base name, path to local files, remote URL):"
)
@ -217,11 +219,11 @@ _shtab_ahriman_package_status_update_options=(
_shtab_ahriman_package_update_options=(
"(- : *)"{-h,--help}"[show this help message and exit]"
{--dependencies,--no-dependencies}"[process missing package dependencies (default\: \%(default)s)]:dependencies:"
{-e,--exit-code}"[return non-zero exit status if result is empty]"
{-n,--now}"[run update function after]"
"*"{-y,--refresh}"[download fresh package databases from the mirror before actions, -yy to force refresh even if up to date]"
{-s,--source}"[explicitly specify the package source for this command]:source:(auto archive aur directory local remote repository)"
"--without-dependencies[do not add dependencies]"
"(*):package source (base name, path to local files, remote URL):"
)
@ -293,6 +295,7 @@ _shtab_ahriman_repo_clean_options=(
_shtab_ahriman_repo_config_options=(
"(- : *)"{-h,--help}"[show this help message and exit]"
{--secure,--no-secure}"[hide passwords and secrets from output (default\: \%(default)s)]:secure:"
)
_shtab_ahriman_repo_config_validate_options=(
@ -304,6 +307,7 @@ _shtab_ahriman_repo_daemon_options=(
"(- : *)"{-h,--help}"[show this help message and exit]"
{-i,--interval}"[interval between runs in seconds]:interval:"
{--aur,--no-aur}"[enable or disable checking for AUR updates (default\: \%(default)s)]:aur:"
{--dependencies,--no-dependencies}"[process missing package dependencies (default\: \%(default)s)]:dependencies:"
{--local,--no-local}"[enable or disable checking of local packages for updates (default\: \%(default)s)]:local:"
{--manual,--no-manual}"[include or exclude manual updates (default\: \%(default)s)]:manual:"
{--vcs,--no-vcs}"[fetch actual version of VCS packages (default\: \%(default)s)]:vcs:"
@ -390,9 +394,10 @@ _shtab_ahriman_repo_triggers_options=(
_shtab_ahriman_repo_update_options=(
"(- : *)"{-h,--help}"[show this help message and exit]"
{--aur,--no-aur}"[enable or disable checking for AUR updates (default\: \%(default)s)]:aur:"
{--dependencies,--no-dependencies}"[process missing package dependencies (default\: \%(default)s)]:dependencies:"
"--dry-run[just perform check for updates, same as check command]"
{-e,--exit-code}"[return non-zero exit status if result is empty]"
{--aur,--no-aur}"[enable or disable checking for AUR updates (default\: \%(default)s)]:aur:"
{--local,--no-local}"[enable or disable checking of local packages for updates (default\: \%(default)s)]:local:"
{--manual,--no-manual}"[include or exclude manual updates (default\: \%(default)s)]:manual:"
{--vcs,--no-vcs}"[fetch actual version of VCS packages (default\: \%(default)s)]:vcs:"
@ -423,6 +428,7 @@ _shtab_ahriman_service_clean_options=(
_shtab_ahriman_service_config_options=(
"(- : *)"{-h,--help}"[show this help message and exit]"
{--secure,--no-secure}"[hide passwords and secrets from output (default\: \%(default)s)]:secure:"
)
_shtab_ahriman_service_config_validate_options=(
@ -504,9 +510,10 @@ _shtab_ahriman_sync_options=(
_shtab_ahriman_update_options=(
"(- : *)"{-h,--help}"[show this help message and exit]"
{--aur,--no-aur}"[enable or disable checking for AUR updates (default\: \%(default)s)]:aur:"
{--dependencies,--no-dependencies}"[process missing package dependencies (default\: \%(default)s)]:dependencies:"
"--dry-run[just perform check for updates, same as check command]"
{-e,--exit-code}"[return non-zero exit status if result is empty]"
{--aur,--no-aur}"[enable or disable checking for AUR updates (default\: \%(default)s)]:aur:"
{--local,--no-local}"[enable or disable checking of local packages for updates (default\: \%(default)s)]:local:"
{--manual,--no-manual}"[include or exclude manual updates (default\: \%(default)s)]:manual:"
{--vcs,--no-vcs}"[fetch actual version of VCS packages (default\: \%(default)s)]:vcs:"

View File

@ -50,6 +50,7 @@ Base authorization settings. ``OAuth`` provider requires ``aioauth-client`` libr
* ``allow_read_only`` - allow requesting status APIs without authorization, boolean, required.
* ``client_id`` - OAuth2 application client ID, string, required in case if ``oauth`` is used.
* ``client_secret`` - OAuth2 application client secret key, string, required in case if ``oauth`` is used.
* ``cookie_secret_key`` - secret key which will be used for cookies encryption, string, optional. It must be 32 url-safe base64-encoded bytes and can be generated as following ``base64.urlsafe_b64encode(os.urandom(32)).decode("utf8")``. If not set, it will be generated automatically; note, however, that in this case, all sessions will be automatically expired during restart.
* ``max_age`` - parameter which controls both cookie expiration and token expiration inside the service, integer, optional, default is 7 days.
* ``oauth_provider`` - OAuth2 provider class name as is in ``aioauth-client`` (e.g. ``GoogleClient``, ``GithubClient`` etc), string, required in case if ``oauth`` is used.
* ``oauth_scopes`` - scopes list for OAuth2 provider, which will allow retrieving user email (which is used for checking user permissions), e.g. ``https://www.googleapis.com/auth/userinfo.email`` for ``GoogleClient`` or ``user:email`` for ``GithubClient``, space separated list of strings, required in case if ``oauth`` is used.
@ -68,7 +69,7 @@ Build related configuration. Group name can refer to architecture, e.g. ``build:
* ``makepkg_flags`` - additional flags passed to ``makepkg`` command, space separated list of strings, optional.
* ``makechrootpkg_flags`` - additional flags passed to ``makechrootpkg`` command, space separated list of strings, optional.
* ``triggers`` - list of ``ahriman.core.triggers.Trigger`` class implementation (e.g. ``ahriman.core.report.ReportTrigger ahriman.core.upload.UploadTrigger``) which will be loaded and run at the end of processing, space separated list of strings, optional. You can also specify triggers by their paths, e.g. ``/usr/lib/python3.10/site-packages/ahriman/core/report/report.py.ReportTrigger``. Triggers are run in the order of mention.
* ``vcs_allowed_age`` - maximal age in seconds of the VCS packages before their version will be updated with its remote source, int, optional, default ``0``.
* ``vcs_allowed_age`` - maximal age in seconds of the VCS packages before their version will be updated with its remote source, int, optional, default ``604800``.
``repository`` group
--------------------

View File

@ -88,4 +88,6 @@ Initial setup
.. code-block:: shell
sudo -u ahriman ahriman -a x86_64 package-add ahriman --now
sudo -u ahriman ahriman -a x86_64 package-add ahriman --now --refresh
The ``--refresh`` flag is required in order to handle local database update.

View File

@ -1,7 +1,7 @@
# Maintainer: Evgeniy Alekseev
pkgname='ahriman'
pkgver=2.6.1
pkgver=2.7.1
pkgrel=1
pkgdesc="ArcH linux ReposItory MANager"
arch=('any')

View File

@ -245,6 +245,8 @@ def _set_package_add_parser(root: SubParserAction) -> argparse.ArgumentParser:
"5) and finally you can add package from AUR.",
formatter_class=_formatter)
parser.add_argument("package", help="package source (base name, path to local files, remote URL)", nargs="+")
parser.add_argument("--dependencies", help="process missing package dependencies",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty", action="store_true")
parser.add_argument("-n", "--now", help="run update function after", action="store_true")
parser.add_argument("-y", "--refresh", help="download fresh package databases from the mirror before actions, "
@ -252,7 +254,6 @@ def _set_package_add_parser(root: SubParserAction) -> argparse.ArgumentParser:
action="count", default=False)
parser.add_argument("-s", "--source", help="explicitly specify the package source for this command",
type=PackageSource, choices=enum_values(PackageSource), default=PackageSource.Auto)
parser.add_argument("--without-dependencies", help="do not add dependencies", action="store_true")
parser.set_defaults(handler=handlers.Add)
return parser
@ -472,7 +473,7 @@ def _set_repo_check_parser(root: SubParserAction) -> argparse.ArgumentParser:
parser.add_argument("-y", "--refresh", help="download fresh package databases from the mirror before actions, "
"-yy to force refresh even if up to date",
action="count", default=False)
parser.set_defaults(handler=handlers.Update, dry_run=True, aur=True, local=True, manual=False)
parser.set_defaults(handler=handlers.Update, dependencies=False, dry_run=True, aur=True, local=True, manual=False)
return parser
@ -492,6 +493,8 @@ def _set_repo_daemon_parser(root: SubParserAction) -> argparse.ArgumentParser:
parser.add_argument("-i", "--interval", help="interval between runs in seconds", type=int, default=60 * 60 * 12)
parser.add_argument("--aur", help="enable or disable checking for AUR updates",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--dependencies", help="process missing package dependencies",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--local", help="enable or disable checking of local packages for updates",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--manual", help="include or exclude manual updates",
@ -691,10 +694,12 @@ def _set_repo_update_parser(root: SubParserAction) -> argparse.ArgumentParser:
description="check for packages updates and run build process if requested",
formatter_class=_formatter)
parser.add_argument("package", help="filter check by package base", nargs="*")
parser.add_argument("--dry-run", help="just perform check for updates, same as check command", action="store_true")
parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty", action="store_true")
parser.add_argument("--aur", help="enable or disable checking for AUR updates",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--dependencies", help="process missing package dependencies",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--dry-run", help="just perform check for updates, same as check command", action="store_true")
parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty", action="store_true")
parser.add_argument("--local", help="enable or disable checking of local packages for updates",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--manual", help="include or exclude manual updates",
@ -750,6 +755,8 @@ def _set_service_config_parser(root: SubParserAction) -> argparse.ArgumentParser
parser = root.add_parser("service-config", aliases=["config", "repo-config"], help="dump configuration",
description="dump configuration for the specified architecture",
formatter_class=_formatter)
parser.add_argument("--secure", help="hide passwords and secrets from output",
action=argparse.BooleanOptionalAction, default=True)
parser.set_defaults(handler=handlers.Dump, lock=None, report=False, quiet=True, unsafe=True)
return parser

View File

@ -17,10 +17,11 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from typing import Set
from typing import Iterable, List, Set
from ahriman.application.application.application_packages import ApplicationPackages
from ahriman.application.application.application_repository import ApplicationRepository
from ahriman.models.package import Package
from ahriman.models.result import Result
@ -87,3 +88,39 @@ class Application(ApplicationPackages, ApplicationRepository):
directly as it will be called after on_start action
"""
self.repository.triggers.on_stop()
def with_dependencies(self, packages: List[Package], *, process_dependencies: bool) -> List[Package]:
"""
add missing dependencies to list of packages
Args:
packages(List[Package]): list of source packages of which dependencies have to be processed
process_dependencies(bool): if no set, dependencies will not be processed
"""
def missing_dependencies(source: Iterable[Package]) -> Set[str]:
# build initial list of dependencies
result = set()
for package in source:
result.update(package.depends_build)
# remove ones which are already well-known
result = result.difference(known_packages)
# remove ones which are in this list already
for package in source:
result = result.difference(package.packages_full)
return result
if not process_dependencies or not packages:
return packages
known_packages = self._known_packages()
with_dependencies = {package.base: package for package in packages}
while missing := missing_dependencies(with_dependencies.values()):
for package_name in missing:
package = Package.from_aur(package_name, self.repository.pacman)
with_dependencies[package.base] = package
return list(with_dependencies.values())

View File

@ -21,7 +21,7 @@ import requests
import shutil
from pathlib import Path
from typing import Any, Iterable, Set
from typing import Any, Iterable
from ahriman.application.application.application_properties import ApplicationProperties
from ahriman.core.build_tools.sources import Sources
@ -47,22 +47,18 @@ class ApplicationPackages(ApplicationProperties):
dst = self.repository.paths.packages / local_path.name
shutil.copy(local_path, dst)
def _add_aur(self, source: str, known_packages: Set[str], without_dependencies: bool) -> None:
def _add_aur(self, source: str) -> None:
"""
add package from AUR
Args:
source(str): package base name
known_packages(Set[str]): list of packages which are known by the service
without_dependencies(bool): if set, dependency check will be disabled
"""
package = Package.from_aur(source, self.repository.pacman)
self.database.build_queue_insert(package)
self.database.remote_update(package)
self._process_dependencies(package, known_packages, without_dependencies)
def _add_directory(self, source: str, *_: Any) -> None:
"""
add packages from directory
@ -74,25 +70,21 @@ class ApplicationPackages(ApplicationProperties):
for full_path in filter(package_like, local_dir.iterdir()):
self._add_archive(str(full_path))
def _add_local(self, source: str, known_packages: Set[str], without_dependencies: bool) -> None:
def _add_local(self, source: str) -> None:
"""
add package from local PKGBUILDs
Args:
source(str): path to directory with local source files
known_packages(Set[str]): list of packages which are known by the service
without_dependencies(bool): if set, dependency check will be disabled
"""
source_dir = Path(source)
package = Package.from_build(source_dir)
package = Package.from_build(source_dir, self.architecture)
cache_dir = self.repository.paths.cache_for(package.base)
shutil.copytree(source_dir, cache_dir) # copy package to store in caches
Sources.init(cache_dir) # we need to run init command in directory where we do have permissions
self.database.build_queue_insert(package)
self._process_dependencies(package, known_packages, without_dependencies)
def _add_remote(self, source: str, *_: Any) -> None:
"""
add package from remote sources (e.g. HTTP)
@ -118,50 +110,19 @@ class ApplicationPackages(ApplicationProperties):
package = Package.from_official(source, self.repository.pacman)
self.database.build_queue_insert(package)
self.database.remote_update(package)
# repository packages must not depend on unknown packages, thus we are not going to process dependencies
def _known_packages(self) -> Set[str]:
"""
load packages from repository and pacman repositories
Returns:
Set[str]: list of known packages
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def _process_dependencies(self, package: Package, known_packages: Set[str], without_dependencies: bool) -> None:
"""
process package dependencies
Args:
package(Package): source package of which dependencies have to be processed
known_packages(Set[str]): list of packages which are known by the service
without_dependencies(bool): if set, dependency check will be disabled
"""
if without_dependencies:
return
dependencies = package.depends_build
self.add(dependencies.difference(known_packages), PackageSource.AUR, without_dependencies)
def add(self, names: Iterable[str], source: PackageSource, without_dependencies: bool) -> None:
def add(self, names: Iterable[str], source: PackageSource) -> None:
"""
add packages for the next build
Args:
names(Iterable[str]): list of package bases to add
source(PackageSource): package source to add
without_dependencies(bool): if set, dependency check will be disabled
"""
known_packages = self._known_packages() # speedup dependencies processing
for name in names:
resolved_source = source.resolve(name)
fn = getattr(self, f"_add_{resolved_source.value}")
fn(name, known_packages, without_dependencies)
fn(name)
def on_result(self, result: Result) -> None:
"""

View File

@ -111,7 +111,7 @@ class ApplicationRepository(ApplicationProperties):
def unknown_local(probe: Package) -> List[str]:
cache_dir = self.repository.paths.cache_for(probe.base)
local = Package.from_build(cache_dir)
local = Package.from_build(cache_dir, self.architecture)
packages = set(probe.packages.keys()).difference(local.packages.keys())
return list(packages)

View File

@ -47,11 +47,12 @@ class Add(Handler):
application = Application(architecture, configuration,
report=report, unsafe=unsafe, refresh_pacman_database=args.refresh)
application.on_start()
application.add(args.package, args.source, args.without_dependencies)
application.add(args.package, args.source)
if not args.now:
return
packages = application.updates(args.package, aur=False, local=False, manual=True, vcs=False,
log_fn=application.logger.info)
packages = application.with_dependencies(packages, process_dependencies=args.dependencies)
result = application.update(packages)
Add.check_if_empty(args.exit_code, result.is_empty)

View File

@ -48,4 +48,4 @@ class Dump(Handler):
"""
dump = configuration.dump()
for section, values in sorted(dump.items()):
ConfigurationPrinter(section, values).print(verbose=False, separator=" = ")
ConfigurationPrinter(section, values).print(verbose=not args.secure, separator=" = ")

View File

@ -58,7 +58,7 @@ class Patch(Handler):
patch = Patch.patch_create_from_function(args.variable, args.patch)
Patch.patch_set_create(application, args.package, patch)
elif args.action == Action.Update and args.variable is None:
package_base, patch = Patch.patch_create_from_diff(args.package, args.track)
package_base, patch = Patch.patch_create_from_diff(args.package, architecture, args.track)
Patch.patch_set_create(application, package_base, patch)
elif args.action == Action.List:
Patch.patch_set_list(application, args.package, args.variable, args.exit_code)
@ -66,19 +66,20 @@ class Patch(Handler):
Patch.patch_set_remove(application, args.package, args.variable)
@staticmethod
def patch_create_from_diff(sources_dir: Path, track: List[str]) -> Tuple[str, PkgbuildPatch]:
def patch_create_from_diff(sources_dir: Path, architecture: str, track: List[str]) -> Tuple[str, PkgbuildPatch]:
"""
create PKGBUILD plain diff patches from sources directory
Args:
sources_dir(Path): path to directory with the package sources
architecture(str): repository architecture
track(List[str]): track files which match the glob before creating the patch
Returns:
Tuple[str, PkgbuildPatch]: package base and created PKGBUILD patch based on the diff from master HEAD
to current files
"""
package = Package.from_build(sources_dir)
package = Package.from_build(sources_dir, architecture)
patch = Sources.patch_create(sources_dir, *track)
return package.base, PkgbuildPatch(None, patch)

View File

@ -40,7 +40,7 @@ class Search(Handler):
"""
ALLOW_AUTO_ARCHITECTURE_RUN = False # it should be called only as "no-architecture"
SORT_FIELDS = {field.name for field in fields(AURPackage) if field.default_factory is not list}
SORT_FIELDS = {field.name for field in fields(AURPackage) if field.default_factory is not list} # type: ignore
@classmethod
def run(cls: Type[Handler], args: argparse.Namespace, architecture: str, configuration: Configuration, *,

View File

@ -53,6 +53,7 @@ class Update(Handler):
if args.dry_run:
return
packages = application.with_dependencies(packages, process_dependencies=args.dependencies)
result = application.update(packages)
Update.check_if_empty(args.exit_code, result.is_empty)

View File

@ -52,7 +52,7 @@ class Validate(Handler):
unsafe(bool): if set no user check will be performed before path creation
"""
schema = Validate.schema(architecture, configuration)
validator = Validator(instance=configuration, schema=schema)
validator = Validator(configuration=configuration, schema=schema)
if validator.validate(configuration.dump()):
return # no errors found

View File

@ -55,3 +55,7 @@ class Web(Handler):
application = setup_service(architecture, configuration, spawner)
run_server(application)
# terminate spawn process at the last
spawner.stop()
spawner.join()

View File

@ -56,7 +56,7 @@ class Configuration(configparser.RawConfigParser):
architecture according to the merge rules. Moreover, the architecture names will be removed from section names.
In order to get current settings, the ``check_loaded`` method can be used. This method will raise an
``InitializeException`` in case if configuration was not yet loaded::
``InitializeError`` in case if configuration was not yet loaded::
>>> path, architecture = configuration.check_loaded()
"""
@ -165,7 +165,7 @@ class Configuration(configparser.RawConfigParser):
Tuple[Path, str]: configuration root path and architecture if loaded
Raises:
InitializeException: in case if architecture and/or path are not set
InitializeError: in case if architecture and/or path are not set
"""
if self.path is None or self.architecture is None:
raise InitializeError("Configuration path and/or architecture are not set")

View File

@ -64,6 +64,7 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"mirror": {
"type": "string",
"required": True,
"is_url": [],
},
"repositories": {
"type": "list",
@ -109,9 +110,15 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"client_secret": {
"type": "string",
},
"cookie_secret_key": {
"type": "string",
"minlength": 32,
"maxlength": 64, # we cannot verify maxlength, because base64 representation might be longer than bytes
},
"max_age": {
"type": "integer",
"coerce": "integer",
"min": 0,
},
"oauth_provider": {
"type": "string",
@ -159,6 +166,7 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"vcs_allowed_age": {
"type": "integer",
"coerce": "integer",
"min": 0,
},
},
},
@ -201,6 +209,7 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"schema": {
"address": {
"type": "string",
"is_url": ["http", "https"],
},
"debug": {
"type": "boolean",
@ -217,9 +226,11 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
},
"host": {
"type": "string",
"is_ip_address": ["localhost"],
},
"index_url": {
"type": "string",
"is_url": ["http", "https"],
},
"password": {
"type": "string",
@ -255,44 +266,4 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
},
},
},
"remote-pull": {
"type": "dict",
"schema": {
"target": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
},
},
},
"remote-push": {
"type": "dict",
"schema": {
"target": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
},
},
},
"report": {
"type": "dict",
"schema": {
"target": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
},
},
},
"upload": {
"type": "dict",
"schema": {
"target": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
},
},
},
}

View File

@ -17,9 +17,12 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import ipaddress
from cerberus import TypeDefinition, Validator as RootValidator # type: ignore
from pathlib import Path
from typing import Any, List
from urllib.parse import urlparse
from ahriman.core.configuration import Configuration
@ -29,7 +32,7 @@ class Validator(RootValidator): # type: ignore
class which defines custom validation methods for the service configuration
Attributes:
instance(Configuration): configuration instance
configuration(Configuration): configuration instance
"""
types_mapping = RootValidator.types_mapping.copy()
@ -40,12 +43,12 @@ class Validator(RootValidator): # type: ignore
default constructor
Args:
instance(Configuration): configuration instance used for extraction
configuration(Configuration): configuration instance used for extraction
*args(Any): positional arguments to be passed to base validator
**kwargs(): keyword arguments to be passed to base validator
"""
RootValidator.__init__(self, *args, **kwargs)
self.instance: Configuration = kwargs["instance"]
self.configuration: Configuration = kwargs["configuration"]
def _normalize_coerce_absolute_path(self, value: str) -> Path:
"""
@ -57,7 +60,7 @@ class Validator(RootValidator): # type: ignore
Returns:
Path: value converted to path instance according to configuration rules
"""
converted: Path = self.instance.converters["path"](value)
converted: Path = self.configuration.converters["path"](value)
return converted
def _normalize_coerce_boolean(self, value: str) -> bool:
@ -71,7 +74,7 @@ class Validator(RootValidator): # type: ignore
bool: value converted to boolean according to configuration rules
"""
# pylint: disable=protected-access
converted: bool = self.instance._convert_to_boolean(value) # type: ignore
converted: bool = self.configuration._convert_to_boolean(value) # type: ignore
return converted
def _normalize_coerce_integer(self, value: str) -> int:
@ -97,9 +100,50 @@ class Validator(RootValidator): # type: ignore
Returns:
List[str]: value converted to string list instance according to configuration rules
"""
converted: List[str] = self.instance.converters["list"](value)
converted: List[str] = self.configuration.converters["list"](value)
return converted
def _validate_is_ip_address(self, constraint: List[str], field: str, value: str) -> None:
"""
check if the specified value is valid ip address
Args:
constraint(List[str]): optional list of allowed special words (e.g. ``localhost``)
field(str): field name to be checked
value(Path): value to be checked
Examples:
The rule's arguments are validated against this schema:
{"type": "list", "schema": {"type": "string"}}
"""
if value in constraint:
return
try:
ipaddress.ip_address(value)
except ValueError:
self._error(field, f"Value {value} must be valid IP address")
def _validate_is_url(self, constraint: List[str], field: str, value: str) -> None:
"""
check if the specified value is a valid url
Args:
constraint(List[str]): optional list of supported schemas. If empty, no schema validation will be performed
field(str): field name to be checked
value(str): value to be checked
Examples:
The rule's arguments are validated against this schema:
{"type": "list", "schema": {"type": "string"}}
"""
url = urlparse(value) # it probably will never rise exceptions on parse
if not url.scheme:
self._error(field, f"Url scheme is not set for {value}")
if not url.netloc and url.scheme not in ("file",):
self._error(field, f"Location must be set for url {value} of scheme {url.scheme}")
if constraint and url.scheme not in constraint:
self._error(field, f"Url {value} scheme must be one of {constraint}")
def _validate_path_exists(self, constraint: bool, field: str, value: Path) -> None:
"""
check if paths exists

View File

@ -114,19 +114,19 @@ def migrate_package_statuses(connection: Connection, paths: RepositoryPaths) ->
values
(:package_base, :version, :aur_url)
""",
dict(package_base=metadata.base, version=metadata.version, aur_url=""))
{"package_base": metadata.base, "version": metadata.version, "aur_url": ""})
connection.execute(
"""
insert into package_statuses
(package_base, status, last_updated)
values
(:package_base, :status, :last_updated)""",
dict(package_base=metadata.base, status=last_status.status.value, last_updated=last_status.timestamp))
{"package_base": metadata.base, "status": last_status.status.value, "last_updated": last_status.timestamp})
def insert_packages(metadata: Package) -> None:
package_list = []
for name, description in metadata.packages.items():
package_list.append(dict(package=name, package_base=metadata.base, **description.view()))
package_list.append({"package": name, "package_base": metadata.base, **description.view()})
connection.executemany(
"""
insert into packages

View File

@ -80,11 +80,11 @@ def migrate_package_remotes(connection: Connection, paths: RepositoryPaths) -> N
web_url = :web_url, source = :source
where package_base = :package_base
""",
dict(
package_base=base,
branch=remote.branch, git_url=remote.git_url, path=remote.path,
web_url=remote.web_url, source=remote.source
)
{
"package_base": base,
"branch": remote.branch, "git_url": remote.git_url, "path": remote.path,
"web_url": remote.web_url, "source": remote.source
}
)
packages = PackageOperations._packages_get_select_package_bases(connection)

View File

@ -71,12 +71,12 @@ class LogsOperations(Operations):
values
(:package_base, :process_id, :created, :record)
""",
dict(
package_base=log_record_id.package_base,
process_id=log_record_id.process_id,
created=created,
record=record
)
{
"package_base": log_record_id.package_base,
"process_id": log_record_id.process_id,
"created": created,
"record": record,
}
)
return self.with_connection(run, commit=True)

View File

@ -82,15 +82,15 @@ class PackageOperations(Operations):
on conflict (package_base) do update set
version = :version, branch = :branch, git_url = :git_url, path = :path, web_url = :web_url, source = :source
""",
dict(
package_base=package.base,
version=package.version,
branch=package.remote.branch if package.remote is not None else None,
git_url=package.remote.git_url if package.remote is not None else None,
path=package.remote.path if package.remote is not None else None,
web_url=package.remote.web_url if package.remote is not None else None,
source=package.remote.source.value if package.remote is not None else None,
)
{
"package_base": package.base,
"version": package.version,
"branch": package.remote.branch if package.remote is not None else None,
"git_url": package.remote.git_url if package.remote is not None else None,
"path": package.remote.path if package.remote is not None else None,
"web_url": package.remote.web_url if package.remote is not None else None,
"source": package.remote.source.value if package.remote is not None else None,
}
)
@staticmethod
@ -106,7 +106,7 @@ class PackageOperations(Operations):
for name, description in package.packages.items():
if description.architecture is None:
continue # architecture is required
package_list.append(dict(package=name, package_base=package.base, **description.view()))
package_list.append({"package": name, "package_base": package.base, **description.view()})
connection.executemany(
"""
insert into packages
@ -145,7 +145,7 @@ class PackageOperations(Operations):
on conflict (package_base) do update set
status = :status, last_updated = :last_updated
""",
dict(package_base=package_base, status=status.status.value, last_updated=status.timestamp))
{"package_base": package_base, "status": status.status.value, "last_updated": status.timestamp})
@staticmethod
def _packages_get_select_package_bases(connection: Connection) -> Dict[str, Package]:

View File

@ -28,9 +28,18 @@ class ConfigurationPrinter(StringPrinter):
print content of the configuration section
Attributes:
HIDE_KEYS(List[str]): (class attribute) hide values for mentioned keys. This list must be used in order to hide
passwords from outputs
values(Dict[str, str]): configuration values dictionary
"""
HIDE_KEYS = [
"api_key", # telegram key
"client_secret", # oauth secret
"password", # generic password (github, email, web server, etc)
"secret_key", # aws secret key
]
def __init__(self, section: str, values: Dict[str, str]) -> None:
"""
default constructor
@ -50,6 +59,6 @@ class ConfigurationPrinter(StringPrinter):
List[Property]: list of content properties
"""
return [
Property(key, value, is_required=True)
Property(key, value, is_required=key not in self.HIDE_KEYS)
for key, value in sorted(self.values.items())
]

View File

@ -33,6 +33,16 @@ class RemotePullTrigger(Trigger):
"""
CONFIGURATION_SCHEMA = {
"remote-pull": {
"type": "dict",
"schema": {
"target": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
},
},
},
"gitremote": {
"type": "dict",
"schema": {

View File

@ -82,7 +82,10 @@ class RemotePush(LazyLogging):
Sources.fetch(package_target_dir, package.remote)
# ...and last, but not least, we remove the dot-git directory...
for git_file in package_target_dir.glob(".git*"):
shutil.rmtree(package_target_dir / git_file)
if git_file.is_file():
git_file.unlink()
else:
shutil.rmtree(git_file)
# ...copy all patches...
for patch in self.database.patches_get(package.base):
filename = f"ahriman-{package.base}.patch" if patch.key is None else f"ahriman-{patch.key}.patch"

View File

@ -38,6 +38,16 @@ class RemotePushTrigger(Trigger):
"""
CONFIGURATION_SCHEMA = {
"remote-push": {
"type": "dict",
"schema": {
"target": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
},
},
},
"gitremote": {
"type": "dict",
"schema": {

View File

@ -35,6 +35,16 @@ class ReportTrigger(Trigger):
"""
CONFIGURATION_SCHEMA = {
"report": {
"type": "dict",
"schema": {
"target": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
},
},
},
"console": {
"type": "dict",
"schema": {
@ -62,6 +72,7 @@ class ReportTrigger(Trigger):
},
"homepage": {
"type": "string",
"is_url": ["http", "https"],
},
"host": {
"type": "string",
@ -70,6 +81,7 @@ class ReportTrigger(Trigger):
"link_path": {
"type": "string",
"required": True,
"is_url": [],
},
"no_empty_report": {
"type": "boolean",
@ -82,6 +94,8 @@ class ReportTrigger(Trigger):
"type": "integer",
"coerce": "integer",
"required": True,
"min": 0,
"max": 65535,
},
"receivers": {
"type": "list",
@ -118,10 +132,12 @@ class ReportTrigger(Trigger):
},
"homepage": {
"type": "string",
"is_url": ["http", "https"],
},
"link_path": {
"type": "string",
"required": True,
"is_url": [],
},
"path": {
"type": "path",
@ -153,10 +169,12 @@ class ReportTrigger(Trigger):
},
"homepage": {
"type": "string",
"is_url": ["http", "https"],
},
"link_path": {
"type": "string",
"required": True,
"is_url": [],
},
"template_path": {
"type": "path",
@ -171,6 +189,7 @@ class ReportTrigger(Trigger):
"timeout": {
"type": "integer",
"coerce": "integer",
"min": 0,
},
},
},

View File

@ -98,7 +98,7 @@ class UpdateHandler(Cleaner):
with self.in_package_context(cache_dir.name):
try:
Sources.fetch(cache_dir, remote=None)
remote = Package.from_build(cache_dir)
remote = Package.from_build(cache_dir, self.architecture)
local = packages.get(remote.base)
if local is None:

View File

@ -60,7 +60,7 @@ class Spawn(Thread, LazyLogging):
self.lock = Lock()
self.active: Dict[str, Process] = {}
# stupid pylint does not know that it is possible
self.queue: Queue[Tuple[str, bool]] = Queue() # pylint: disable=unsubscriptable-object
self.queue: Queue[Tuple[str, bool] | None] = Queue() # pylint: disable=unsubscriptable-object
@staticmethod
def process(callback: Callable[[argparse.Namespace, str], bool], args: argparse.Namespace, architecture: str,
@ -78,55 +78,7 @@ class Spawn(Thread, LazyLogging):
result = callback(args, architecture)
queue.put((process_id, result))
def key_import(self, key: str, server: Optional[str]) -> None:
"""
import key to service cache
Args:
key(str): key to import
server(str): PGP key server
"""
kwargs = {} if server is None else {"key-server": server}
self.spawn_process("service-key-import", key, **kwargs)
def packages_add(self, packages: Iterable[str], *, now: bool) -> None:
"""
add packages
Args:
packages(Iterable[str]): packages list to add
now(bool): build packages now
"""
kwargs = {"source": PackageSource.AUR.value} # avoid abusing by building non-aur packages
if now:
kwargs["now"] = ""
self.spawn_process("package-add", *packages, **kwargs)
def packages_rebuild(self, depends_on: str) -> None:
"""
rebuild packages which depend on the specified package
Args:
depends_on(str): packages dependency
"""
self.spawn_process("repo-rebuild", **{"depends-on": depends_on})
def packages_remove(self, packages: Iterable[str]) -> None:
"""
remove packages
Args:
packages(Iterable[str]): packages list to remove
"""
self.spawn_process("package-remove", *packages)
def packages_update(self, ) -> None:
"""
run full repository update
"""
self.spawn_process("repo-update")
def spawn_process(self, command: str, *args: str, **kwargs: str) -> None:
def _spawn_process(self, command: str, *args: str, **kwargs: str) -> None:
"""
spawn external ahriman process with supplied arguments
@ -161,6 +113,54 @@ class Spawn(Thread, LazyLogging):
with self.lock:
self.active[process_id] = process
def key_import(self, key: str, server: Optional[str]) -> None:
"""
import key to service cache
Args:
key(str): key to import
server(str): PGP key server
"""
kwargs = {} if server is None else {"key-server": server}
self._spawn_process("service-key-import", key, **kwargs)
def packages_add(self, packages: Iterable[str], *, now: bool) -> None:
"""
add packages
Args:
packages(Iterable[str]): packages list to add
now(bool): build packages now
"""
kwargs = {"source": PackageSource.AUR.value} # avoid abusing by building non-aur packages
if now:
kwargs["now"] = ""
self._spawn_process("package-add", *packages, **kwargs)
def packages_rebuild(self, depends_on: str) -> None:
"""
rebuild packages which depend on the specified package
Args:
depends_on(str): packages dependency
"""
self._spawn_process("repo-rebuild", **{"depends-on": depends_on})
def packages_remove(self, packages: Iterable[str]) -> None:
"""
remove packages
Args:
packages(Iterable[str]): packages list to remove
"""
self._spawn_process("package-remove", *packages)
def packages_update(self, ) -> None:
"""
run full repository update
"""
self._spawn_process("repo-update")
def run(self) -> None:
"""
thread run method
@ -174,3 +174,9 @@ class Spawn(Thread, LazyLogging):
if process is not None:
process.terminate() # make sure lol
process.join()
def stop(self) -> None:
"""
gracefully terminate thread
"""
self.queue.put(None)

View File

@ -35,6 +35,16 @@ class UploadTrigger(Trigger):
"""
CONFIGURATION_SCHEMA = {
"upload": {
"type": "dict",
"schema": {
"target": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
},
},
},
"github": {
"type": "dict",
"schema": {
@ -57,6 +67,7 @@ class UploadTrigger(Trigger):
"timeout": {
"type": "integer",
"coerce": "integer",
"min": 0,
},
"username": {
"type": "string",
@ -101,6 +112,7 @@ class UploadTrigger(Trigger):
"chunk_size": {
"type": "integer",
"coerce": "integer",
"min": 0,
},
"region": {
"type": "string",

View File

@ -17,7 +17,7 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# pylint: disable=too-many-lines
# pylint: disable=too-many-lines,too-many-public-methods
from __future__ import annotations
import copy
@ -96,7 +96,7 @@ class Package(LazyLogging):
Returns:
Set[str]: full dependencies list used by devtools
"""
return (set(self.depends) | set(self.depends_make)) - self.packages.keys()
return (set(self.depends) | set(self.depends_make)).difference(self.packages_full)
@property
def depends_make(self) -> List[str]:
@ -163,6 +163,20 @@ class Package(LazyLogging):
"""
return sorted(set(sum((package.licenses for package in self.packages.values()), start=[])))
@property
def packages_full(self) -> List[str]:
"""
get full packages list including provides
Returns:
List[str]: full list of packages which this base contains
"""
packages = set()
for package, properties in self.packages.items():
packages.add(package)
packages.update(properties.provides)
return sorted(packages)
@classmethod
def from_archive(cls: Type[Package], path: Path, pacman: Pacman, remote: Optional[RemoteSource]) -> Package:
"""
@ -201,12 +215,13 @@ class Package(LazyLogging):
packages={package.name: PackageDescription.from_aur(package)})
@classmethod
def from_build(cls: Type[Package], path: Path) -> Package:
def from_build(cls: Type[Package], path: Path, architecture: str) -> Package:
"""
construct package properties from sources directory
Args:
path(Path): path to package sources directory
architecture(str): load package for specific architecture
Returns:
Package: package properties
@ -220,13 +235,16 @@ class Package(LazyLogging):
raise PackageInfoError(errors)
def get_property(key: str, properties: Dict[str, Any], default: Any) -> Any:
return properties.get(key, srcinfo.get(key, default))
return properties.get(key) or srcinfo.get(key) or default
def get_list(key: str, properties: Dict[str, Any]) -> Any:
return get_property(key, properties, []) + get_property(f"{key}_{architecture}", properties, [])
packages = {
package: PackageDescription(
depends=get_property("depends", properties, []),
make_depends=get_property("makedepends", properties, []),
opt_depends=get_property("optdepends", properties, []),
depends=get_list("depends", properties),
make_depends=get_list("makedepends", properties),
opt_depends=get_list("optdepends", properties),
)
for package, properties in srcinfo["packages"].items()
}

View File

@ -1,102 +0,0 @@
#
# Copyright (c) 2021-2023 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from __future__ import annotations
import time
from dataclasses import dataclass
from typing import Optional, Type
@dataclass(frozen=True)
class UserIdentity:
"""
user identity used inside web service
Attributes:
username(str): username
expire_at(int): identity expiration timestamp
"""
username: str
expire_at: int
@classmethod
def from_identity(cls: Type[UserIdentity], identity: str) -> Optional[UserIdentity]:
"""
parse identity into object
Args:
identity(str): identity from session data
Returns:
Optional[UserIdentity]: user identity object if it can be parsed and not expired and None otherwise
"""
try:
username, expire_at = identity.split()
user = cls(username, int(expire_at))
return None if user.is_expired() else user
except ValueError:
return None
@classmethod
def from_username(cls: Type[UserIdentity], username: Optional[str], max_age: int) -> Optional[UserIdentity]:
"""
generate identity from username
Args:
username(Optional[str]): username
max_age(int): time to expire, seconds
Returns:
Optional[UserIdentity]: constructed identity object
"""
return cls(username, cls.expire_when(max_age)) if username is not None else None
@staticmethod
def expire_when(max_age: int) -> int:
"""
generate expiration time using delta
Args:
max_age(int): time delta to generate. Must be usually TTE
Returns:
int: expiration timestamp
"""
return int(time.time()) + max_age
def is_expired(self) -> bool:
"""
compare timestamp with current timestamp and return True in case if identity is expired
Returns:
bool: True in case if identity is expired and False otherwise
"""
return self.expire_when(0) > self.expire_at
def to_identity(self) -> str:
"""
convert object to identity representation
Returns:
str: web service identity
"""
return f"{self.username} {self.expire_at}"

View File

@ -17,4 +17,4 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
__version__ = "2.6.1"
__version__ = "2.7.1"

View File

@ -18,7 +18,6 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import aiohttp_security # type: ignore
import base64
import socket
import types
@ -32,12 +31,12 @@ from cryptography import fernet
from typing import Optional
from ahriman.core.auth import Auth
from ahriman.core.configuration import Configuration
from ahriman.models.user_access import UserAccess
from ahriman.models.user_identity import UserIdentity
from ahriman.web.middlewares import HandlerType, MiddlewareType
__all__ = ["AuthorizationPolicy", "auth_handler", "setup_auth"]
__all__ = ["AuthorizationPolicy", "auth_handler", "cookie_secret_key", "setup_auth"]
class AuthorizationPolicy(aiohttp_security.AbstractAuthorizationPolicy): # type: ignore
@ -67,10 +66,7 @@ class AuthorizationPolicy(aiohttp_security.AbstractAuthorizationPolicy): # type
Returns:
Optional[str]: user identity (username) in case if user exists and None otherwise
"""
user = UserIdentity.from_identity(identity)
if user is None:
return None
return user.username if await self.validator.known_username(user.username) else None
return identity if await self.validator.known_username(identity) else None
async def permits(self, identity: str, permission: UserAccess, context: Optional[str] = None) -> bool:
"""
@ -84,10 +80,7 @@ class AuthorizationPolicy(aiohttp_security.AbstractAuthorizationPolicy): # type
Returns:
bool: True in case if user is allowed to perform this request and False otherwise
"""
user = UserIdentity.from_identity(identity)
if user is None:
return False
return await self.validator.verify_access(user.username, permission, context)
return await self.validator.verify_access(identity, permission, context)
def auth_handler(allow_read_only: bool) -> MiddlewareType:
@ -125,19 +118,36 @@ def auth_handler(allow_read_only: bool) -> MiddlewareType:
return handle
def setup_auth(application: web.Application, validator: Auth) -> web.Application:
def cookie_secret_key(configuration: Configuration) -> fernet.Fernet:
"""
extract cookie secret key from configuration if set or generate new one
Args:
configuration(Configuration): configuration instance
Returns:
fernet.Fernet: fernet key instance
"""
if (secret_key := configuration.get("auth", "cookie_secret_key", fallback=None)) is not None:
return fernet.Fernet(secret_key)
secret_key = fernet.Fernet.generate_key()
return fernet.Fernet(secret_key)
def setup_auth(application: web.Application, configuration: Configuration, validator: Auth) -> web.Application:
"""
setup authorization policies for the application
Args:
application(web.Application): web application instance
configuration(Configuration): configuration instance
validator(Auth): authorization module instance
Returns:
web.Application: configured web application
"""
fernet_key = fernet.Fernet.generate_key()
secret_key = base64.urlsafe_b64decode(fernet_key)
secret_key = cookie_secret_key(configuration)
storage = EncryptedCookieStorage(secret_key, cookie_name="API_SESSION", max_age=validator.max_age)
setup_session(application, storage)

View File

@ -21,7 +21,6 @@ from aiohttp.web import HTTPFound, HTTPMethodNotAllowed, HTTPUnauthorized
from ahriman.core.auth.helpers import remember
from ahriman.models.user_access import UserAccess
from ahriman.models.user_identity import UserIdentity
from ahriman.web.views.base import BaseView
@ -64,10 +63,9 @@ class LoginView(BaseView):
raise HTTPFound(oauth_provider.get_oauth_url())
response = HTTPFound("/")
username = await oauth_provider.get_oauth_username(code)
identity = UserIdentity.from_username(username, self.validator.max_age)
if identity is not None and await self.validator.known_username(username):
await remember(self.request, response, identity.to_identity())
identity = await oauth_provider.get_oauth_username(code)
if identity is not None and await self.validator.known_username(identity):
await remember(self.request, response, identity)
raise response
raise HTTPUnauthorized()
@ -111,12 +109,11 @@ class LoginView(BaseView):
302: Found
"""
data = await self.extract_data()
username = data.get("username")
identity = data.get("username")
response = HTTPFound("/")
identity = UserIdentity.from_username(username, self.validator.max_age)
if identity is not None and await self.validator.check_credentials(username, data.get("password")):
await remember(self.request, response, identity.to_identity())
if identity is not None and await self.validator.check_credentials(identity, data.get("password")):
await remember(self.request, response, identity)
raise response
raise HTTPUnauthorized()

View File

@ -90,7 +90,7 @@ async def on_startup(application: web.Application) -> None:
application(web.Application): web application instance
Raises:
InitializeException: in case if matched could not be loaded
InitializeError: in case if matched could not be loaded
"""
application.logger.info("server started")
try:
@ -115,7 +115,7 @@ def run_server(application: web.Application) -> None:
port = configuration.getint("web", "port")
unix_socket = create_socket(configuration, application)
web.run_app(application, host=host, port=port, sock=unix_socket, handle_signals=False,
web.run_app(application, host=host, port=port, sock=unix_socket, handle_signals=True,
access_log=logging.getLogger("http"), access_log_class=FilteredAccessLogger)
@ -168,6 +168,6 @@ def setup_service(architecture: str, configuration: Configuration, spawner: Spaw
validator = application["validator"] = Auth.load(configuration, database)
if validator.enabled:
from ahriman.web.middlewares.auth_handler import setup_auth
setup_auth(application, validator)
setup_auth(application, configuration, validator)
return application

View File

@ -1,4 +1,5 @@
from pytest_mock import MockerFixture
from unittest.mock import MagicMock, call as MockCall
from ahriman.application.application import Application
from ahriman.models.package import Package
@ -44,3 +45,55 @@ def test_on_stop(application: Application, mocker: MockerFixture) -> None:
application.on_stop()
triggers_mock.assert_called_once_with()
def test_with_dependencies(application: Application, package_ahriman: Package, package_python_schedule: Package,
mocker: MockerFixture) -> None:
"""
must append list of missing dependencies
"""
def create_package_mock(package_base) -> MagicMock:
mock = MagicMock()
mock.base = package_base
mock.depends_build = []
mock.packages_full = [package_base]
return mock
package_python_schedule.packages = {
package_python_schedule.base: package_python_schedule.packages[package_python_schedule.base]
}
package_ahriman.packages[package_ahriman.base].depends = ["devtools", "python", package_python_schedule.base]
package_ahriman.packages[package_ahriman.base].make_depends = ["python-build", "python-installer"]
packages = {
package_ahriman.base: package_ahriman,
package_python_schedule.base: package_python_schedule,
"python": create_package_mock("python"),
"python-installer": create_package_mock("python-installer"),
}
package_mock = mocker.patch("ahriman.models.package.Package.from_aur", side_effect=lambda p, _: packages[p])
packages_mock = mocker.patch("ahriman.application.application.Application._known_packages",
return_value=["devtools", "python-build"])
result = application.with_dependencies([package_ahriman], process_dependencies=True)
assert {package.base: package for package in result} == packages
package_mock.assert_has_calls([
MockCall(package_python_schedule.base, application.repository.pacman),
MockCall("python", application.repository.pacman),
MockCall("python-installer", application.repository.pacman),
], any_order=True)
packages_mock.assert_called_once_with()
def test_with_dependencies_skip(application: Application, package_ahriman: Package, mocker: MockerFixture) -> None:
"""
must skip processing of dependencies
"""
packages_mock = mocker.patch("ahriman.application.application.Application._known_packages")
assert application.with_dependencies([package_ahriman], process_dependencies=False) == [package_ahriman]
packages_mock.assert_not_called()
assert application.with_dependencies([], process_dependencies=True) == []
packages_mock.assert_not_called()

View File

@ -29,13 +29,10 @@ def test_add_aur(application_packages: ApplicationPackages, package_ahriman: Pac
must add package from AUR
"""
mocker.patch("ahriman.models.package.Package.from_aur", return_value=package_ahriman)
dependencies_mock = mocker.patch(
"ahriman.application.application.application_packages.ApplicationPackages._process_dependencies")
build_queue_mock = mocker.patch("ahriman.core.database.SQLite.build_queue_insert")
update_remote_mock = mocker.patch("ahriman.core.database.SQLite.remote_update")
application_packages._add_aur(package_ahriman.base, set(), False)
dependencies_mock.assert_called_once_with(pytest.helpers.anyvar(int), set(), False)
application_packages._add_aur(package_ahriman.base)
build_queue_mock.assert_called_once_with(package_ahriman)
update_remote_mock.assert_called_once_with(package_ahriman)
@ -64,15 +61,12 @@ def test_add_local(application_packages: ApplicationPackages, package_ahriman: P
mocker.patch("ahriman.models.package.Package.from_build", return_value=package_ahriman)
init_mock = mocker.patch("ahriman.core.build_tools.sources.Sources.init")
copytree_mock = mocker.patch("shutil.copytree")
dependencies_mock = mocker.patch(
"ahriman.application.application.application_packages.ApplicationPackages._process_dependencies")
build_queue_mock = mocker.patch("ahriman.core.database.SQLite.build_queue_insert")
application_packages._add_local(package_ahriman.base, set(), False)
application_packages._add_local(package_ahriman.base)
copytree_mock.assert_called_once_with(
Path(package_ahriman.base), application_packages.repository.paths.cache_for(package_ahriman.base))
init_mock.assert_called_once_with(application_packages.repository.paths.cache_for(package_ahriman.base))
dependencies_mock.assert_called_once_with(pytest.helpers.anyvar(int), set(), False)
build_queue_mock.assert_called_once_with(package_ahriman)
@ -107,59 +101,15 @@ def test_add_repository(application_packages: ApplicationPackages, package_ahrim
update_remote_mock.assert_called_once_with(package_ahriman)
def test_known_packages(application_packages: ApplicationPackages) -> None:
"""
must raise NotImplemented for missing known_packages method
"""
with pytest.raises(NotImplementedError):
application_packages._known_packages()
def test_process_dependencies(application_packages: ApplicationPackages, package_ahriman: Package,
mocker: MockerFixture) -> None:
"""
must process dependencies addition
"""
add_mock = mocker.patch("ahriman.application.application.application_packages.ApplicationPackages.add")
application_packages._process_dependencies(package_ahriman, set(), False)
add_mock.assert_called_once_with(package_ahriman.depends_build, PackageSource.AUR, False)
def test_process_dependencies_missing(application_packages: ApplicationPackages, package_ahriman: Package,
mocker: MockerFixture) -> None:
"""
must process dependencies addition only for missing packages
"""
missing = {"devtools"}
add_mock = mocker.patch("ahriman.application.application.application_packages.ApplicationPackages.add")
application_packages._process_dependencies(
package_ahriman, package_ahriman.depends_build.difference(missing), False)
add_mock.assert_called_once_with(missing, PackageSource.AUR, False)
def test_process_dependencies_skip(application_packages: ApplicationPackages, package_ahriman: Package,
mocker: MockerFixture) -> None:
"""
must skip dependencies processing
"""
add_mock = mocker.patch("ahriman.application.application.application_packages.ApplicationPackages.add")
application_packages._process_dependencies(package_ahriman, set(), True)
add_mock.assert_not_called()
def test_add_add_archive(application_packages: ApplicationPackages, package_ahriman: Package,
mocker: MockerFixture) -> None:
"""
must add package from archive via add function
"""
mocker.patch("ahriman.application.application.application_packages.ApplicationPackages._known_packages",
return_value=set())
add_mock = mocker.patch("ahriman.application.application.application_packages.ApplicationPackages._add_archive")
application_packages.add([package_ahriman.base], PackageSource.Archive, False)
add_mock.assert_called_once_with(package_ahriman.base, set(), False)
application_packages.add([package_ahriman.base], PackageSource.Archive)
add_mock.assert_called_once_with(package_ahriman.base)
def test_add_add_aur(
@ -169,12 +119,10 @@ def test_add_add_aur(
"""
must add package from AUR via add function
"""
mocker.patch("ahriman.application.application.application_packages.ApplicationPackages._known_packages",
return_value=set())
add_mock = mocker.patch("ahriman.application.application.application_packages.ApplicationPackages._add_aur")
application_packages.add([package_ahriman.base], PackageSource.AUR, True)
add_mock.assert_called_once_with(package_ahriman.base, set(), True)
application_packages.add([package_ahriman.base], PackageSource.AUR)
add_mock.assert_called_once_with(package_ahriman.base)
def test_add_add_directory(application_packages: ApplicationPackages, package_ahriman: Package,
@ -182,12 +130,10 @@ def test_add_add_directory(application_packages: ApplicationPackages, package_ah
"""
must add packages from directory via add function
"""
mocker.patch("ahriman.application.application.application_packages.ApplicationPackages._known_packages",
return_value=set())
add_mock = mocker.patch("ahriman.application.application.application_packages.ApplicationPackages._add_directory")
application_packages.add([package_ahriman.base], PackageSource.Directory, False)
add_mock.assert_called_once_with(package_ahriman.base, set(), False)
application_packages.add([package_ahriman.base], PackageSource.Directory)
add_mock.assert_called_once_with(package_ahriman.base)
def test_add_add_local(application_packages: ApplicationPackages, package_ahriman: Package,
@ -195,12 +141,10 @@ def test_add_add_local(application_packages: ApplicationPackages, package_ahrima
"""
must add package from local sources via add function
"""
mocker.patch("ahriman.application.application.application_packages.ApplicationPackages._known_packages",
return_value=set())
add_mock = mocker.patch("ahriman.application.application.application_packages.ApplicationPackages._add_local")
application_packages.add([package_ahriman.base], PackageSource.Local, False)
add_mock.assert_called_once_with(package_ahriman.base, set(), False)
application_packages.add([package_ahriman.base], PackageSource.Local)
add_mock.assert_called_once_with(package_ahriman.base)
def test_add_add_remote(application_packages: ApplicationPackages, package_description_ahriman: PackageDescription,
@ -208,13 +152,11 @@ def test_add_add_remote(application_packages: ApplicationPackages, package_descr
"""
must add package from remote source via add function
"""
mocker.patch("ahriman.application.application.application_packages.ApplicationPackages._known_packages",
return_value=set())
add_mock = mocker.patch("ahriman.application.application.application_packages.ApplicationPackages._add_remote")
url = f"https://host/{package_description_ahriman.filename}"
application_packages.add([url], PackageSource.Remote, False)
add_mock.assert_called_once_with(url, set(), False)
application_packages.add([url], PackageSource.Remote)
add_mock.assert_called_once_with(url)
def test_on_result(application_packages: ApplicationPackages) -> None:

View File

@ -26,7 +26,7 @@ def _default_args(args: argparse.Namespace) -> argparse.Namespace:
args.now = False
args.refresh = 0
args.source = PackageSource.Auto
args.without_dependencies = False
args.dependencies = True
return args
@ -38,10 +38,12 @@ def test_run(args: argparse.Namespace, configuration: Configuration, repository:
args = _default_args(args)
mocker.patch("ahriman.core.repository.Repository.load", return_value=repository)
application_mock = mocker.patch("ahriman.application.application.Application.add")
dependencies_mock = mocker.patch("ahriman.application.application.Application.with_dependencies")
on_start_mock = mocker.patch("ahriman.application.application.Application.on_start")
Add.run(args, "x86_64", configuration, report=False, unsafe=False)
application_mock.assert_called_once_with(args.package, args.source, args.without_dependencies)
application_mock.assert_called_once_with(args.package, args.source)
dependencies_mock.assert_not_called()
on_start_mock.assert_called_once_with()
@ -59,11 +61,14 @@ def test_run_with_updates(args: argparse.Namespace, configuration: Configuration
application_mock = mocker.patch("ahriman.application.application.Application.update", return_value=result)
check_mock = mocker.patch("ahriman.application.handlers.Handler.check_if_empty")
updates_mock = mocker.patch("ahriman.application.application.Application.updates", return_value=[package_ahriman])
dependencies_mock = mocker.patch("ahriman.application.application.Application.with_dependencies",
return_value=[package_ahriman])
Add.run(args, "x86_64", configuration, report=False, unsafe=False)
updates_mock.assert_called_once_with(args.package, aur=False, local=False, manual=True, vcs=False,
log_fn=pytest.helpers.anyvar(int))
application_mock.assert_called_once_with([package_ahriman])
dependencies_mock.assert_called_once_with([package_ahriman], process_dependencies=args.dependencies)
check_mock.assert_called_once_with(False, False)
@ -78,6 +83,7 @@ def test_run_empty_exception(args: argparse.Namespace, configuration: Configurat
mocker.patch("ahriman.application.application.Application.add")
mocker.patch("ahriman.core.repository.Repository.load", return_value=repository)
mocker.patch("ahriman.application.application.Application.update", return_value=Result())
mocker.patch("ahriman.application.application.Application.with_dependencies")
mocker.patch("ahriman.application.application.Application.updates")
check_mock = mocker.patch("ahriman.application.handlers.Handler.check_if_empty")

View File

@ -6,10 +6,25 @@ from ahriman.application.handlers import Dump
from ahriman.core.configuration import Configuration
def _default_args(args: argparse.Namespace) -> argparse.Namespace:
"""
default arguments for these test cases
Args:
args(argparse.Namespace): command line arguments fixture
Returns:
argparse.Namespace: generated arguments for these test cases
"""
args.secure = True
return args
def test_run(args: argparse.Namespace, configuration: Configuration, mocker: MockerFixture) -> None:
"""
must run command
"""
args = _default_args(args)
print_mock = mocker.patch("ahriman.core.formatters.Printer.print")
application_mock = mocker.patch("ahriman.core.configuration.Configuration.dump",
return_value=configuration.dump())

View File

@ -45,7 +45,7 @@ def test_run(args: argparse.Namespace, configuration: Configuration, repository:
application_mock = mocker.patch("ahriman.application.handlers.Patch.patch_set_create")
Patch.run(args, "x86_64", configuration, report=False, unsafe=False)
patch_mock.assert_called_once_with(args.package, args.track)
patch_mock.assert_called_once_with(args.package, "x86_64", args.track)
application_mock.assert_called_once_with(pytest.helpers.anyvar(int), args.package, PkgbuildPatch(None, "patch"))
@ -108,8 +108,8 @@ def test_patch_create_from_diff(package_ahriman: Package, mocker: MockerFixture)
package_mock = mocker.patch("ahriman.models.package.Package.from_build", return_value=package_ahriman)
sources_mock = mocker.patch("ahriman.core.build_tools.sources.Sources.patch_create", return_value=patch.value)
assert Patch.patch_create_from_diff(path, ["*.diff"]) == (package_ahriman.base, patch)
package_mock.assert_called_once_with(path)
assert Patch.patch_create_from_diff(path, "x86_64", ["*.diff"]) == (package_ahriman.base, patch)
package_mock.assert_called_once_with(path, "x86_64")
sources_mock.assert_called_once_with(path, "*.diff")

View File

@ -131,9 +131,10 @@ def test_disallow_auto_architecture_run() -> None:
assert not Search.ALLOW_AUTO_ARCHITECTURE_RUN
def test_sort_fields() -> None:
def test_sort_fields(aur_package_ahriman: AURPackage) -> None:
"""
must store valid field list which are allowed to be used for sorting
"""
expected = {field.name for field in dataclasses.fields(AURPackage)}
assert all(field in expected for field in Search.SORT_FIELDS)
assert all(not isinstance(getattr(aur_package_ahriman, field), list) for field in Search.SORT_FIELDS)

View File

@ -23,6 +23,7 @@ def _default_args(args: argparse.Namespace) -> argparse.Namespace:
argparse.Namespace: generated arguments for these test cases
"""
args.package = []
args.dependencies = True
args.dry_run = False
args.exit_code = False
args.aur = True
@ -44,6 +45,8 @@ def test_run(args: argparse.Namespace, package_ahriman: Package, configuration:
mocker.patch("ahriman.core.repository.Repository.load", return_value=repository)
application_mock = mocker.patch("ahriman.application.application.Application.update", return_value=result)
check_mock = mocker.patch("ahriman.application.handlers.Handler.check_if_empty")
dependencies_mock = mocker.patch("ahriman.application.application.Application.with_dependencies",
return_value=[package_ahriman])
updates_mock = mocker.patch("ahriman.application.application.Application.updates", return_value=[package_ahriman])
on_start_mock = mocker.patch("ahriman.application.application.Application.on_start")
@ -51,6 +54,7 @@ def test_run(args: argparse.Namespace, package_ahriman: Package, configuration:
application_mock.assert_called_once_with([package_ahriman])
updates_mock.assert_called_once_with(args.package, aur=args.aur, local=args.local, manual=args.manual, vcs=args.vcs,
log_fn=pytest.helpers.anyvar(int))
dependencies_mock.assert_called_once_with([package_ahriman], process_dependencies=args.dependencies)
check_mock.assert_has_calls([MockCall(False, False), MockCall(False, False)])
on_start_mock.assert_called_once_with()
@ -81,6 +85,7 @@ def test_run_update_empty_exception(args: argparse.Namespace, package_ahriman: P
mocker.patch("ahriman.core.repository.Repository.load", return_value=repository)
mocker.patch("ahriman.application.application.Application.update", return_value=Result())
mocker.patch("ahriman.application.application.Application.updates", return_value=[package_ahriman])
mocker.patch("ahriman.application.application.Application.with_dependencies", return_value=[package_ahriman])
check_mock = mocker.patch("ahriman.application.handlers.Handler.check_if_empty")
Update.run(args, "x86_64", configuration, report=False, unsafe=False)

View File

@ -62,9 +62,11 @@ def test_schema(configuration: Configuration) -> None:
assert schema.pop("email")
assert schema.pop("github")
assert schema.pop("html")
assert schema.pop("report")
assert schema.pop("rsync")
assert schema.pop("s3")
assert schema.pop("telegram")
assert schema.pop("upload")
assert schema == CONFIGURATION_SCHEMA

View File

@ -28,14 +28,19 @@ def test_run(args: argparse.Namespace, configuration: Configuration, repository:
must run command
"""
args = _default_args(args)
mocker.patch("ahriman.core.spawn.Spawn.start")
mocker.patch("ahriman.core.repository.Repository.load", return_value=repository)
setup_mock = mocker.patch("ahriman.web.web.setup_service")
run_mock = mocker.patch("ahriman.web.web.run_server")
start_mock = mocker.patch("ahriman.core.spawn.Spawn.start")
stop_mock = mocker.patch("ahriman.core.spawn.Spawn.stop")
join_mock = mocker.patch("ahriman.core.spawn.Spawn.join")
Web.run(args, "x86_64", configuration, report=False, unsafe=False)
setup_mock.assert_called_once_with("x86_64", configuration, pytest.helpers.anyvar(int))
run_mock.assert_called_once_with(pytest.helpers.anyvar(int))
start_mock.assert_called_once_with()
stop_mock.assert_called_once_with()
join_mock.assert_called_once_with()
def test_disallow_auto_architecture_run() -> None:

View File

@ -342,9 +342,10 @@ def test_subparsers_repo_backup_architecture(parser: argparse.ArgumentParser) ->
def test_subparsers_repo_check(parser: argparse.ArgumentParser) -> None:
"""
repo-check command must imply dry-run, aur and manual
repo-check command must imply dependencies, dry-run, aur and manual
"""
args = parser.parse_args(["repo-check"])
assert not args.dependencies
assert args.dry_run
assert args.aur
assert not args.manual

View File

@ -16,4 +16,4 @@ def validator(configuration: Configuration) -> Validator:
Returns:
Validator: validator test instance
"""
return Validator(instance=configuration, schema=CONFIGURATION_SCHEMA)
return Validator(configuration=configuration, schema=CONFIGURATION_SCHEMA)

View File

@ -1,6 +1,6 @@
from pathlib import Path
from pytest_mock import MockerFixture
from unittest.mock import MagicMock
from unittest.mock import MagicMock, call as MockCall
from ahriman.core.configuration.validator import Validator
@ -18,7 +18,7 @@ def test_normalize_coerce_absolute_path(validator: Validator) -> None:
must convert string value to path by using configuration converters
"""
convert_mock = MagicMock()
validator.instance.converters["path"] = convert_mock
validator.configuration.converters["path"] = convert_mock
validator._normalize_coerce_absolute_path("value")
convert_mock.assert_called_once_with("value")
@ -46,12 +46,56 @@ def test_normalize_coerce_list(validator: Validator) -> None:
must convert string value to list by using configuration converters
"""
convert_mock = MagicMock()
validator.instance.converters["list"] = convert_mock
validator.configuration.converters["list"] = convert_mock
validator._normalize_coerce_list("value")
convert_mock.assert_called_once_with("value")
def test_validate_is_ip_address(validator: Validator, mocker: MockerFixture) -> None:
"""
must validate addresses correctly
"""
error_mock = mocker.patch("ahriman.core.configuration.validator.Validator._error")
validator._validate_is_ip_address(["localhost"], "field", "localhost")
validator._validate_is_ip_address([], "field", "localhost")
validator._validate_is_ip_address([], "field", "127.0.0.1")
validator._validate_is_ip_address([], "field", "::")
validator._validate_is_ip_address([], "field", "0.0.0.0")
validator._validate_is_ip_address([], "field", "random string")
error_mock.assert_has_calls([
MockCall("field", "Value localhost must be valid IP address"),
MockCall("field", "Value random string must be valid IP address"),
])
def test_validate_is_url(validator: Validator, mocker: MockerFixture) -> None:
"""
must validate url correctly
"""
error_mock = mocker.patch("ahriman.core.configuration.validator.Validator._error")
validator._validate_is_url([], "field", "http://example.com")
validator._validate_is_url([], "field", "https://example.com")
validator._validate_is_url([], "field", "file:///tmp")
validator._validate_is_url(["http", "https"], "field", "file:///tmp")
validator._validate_is_url([], "field", "http:///path")
validator._validate_is_url([], "field", "random string")
error_mock.assert_has_calls([
MockCall("field", "Url file:///tmp scheme must be one of ['http', 'https']"),
MockCall("field", "Location must be set for url http:///path of scheme http"),
MockCall("field", "Url scheme is not set for random string"),
])
def test_validate_path_exists(validator: Validator, mocker: MockerFixture) -> None:
"""
must validate that paths exists
@ -67,4 +111,6 @@ def test_validate_path_exists(validator: Validator, mocker: MockerFixture) -> No
mocker.patch("pathlib.Path.exists", return_value=True)
validator._validate_path_exists(True, "field", Path("3"))
error_mock.assert_called_once_with("field", "Path 2 must exist")
error_mock.assert_has_calls([
MockCall("field", "Path 2 must exist"),
])

View File

@ -10,10 +10,13 @@ def test_properties(configuration_printer: ConfigurationPrinter) -> None:
def test_properties_required(configuration_printer: ConfigurationPrinter) -> None:
"""
must return all properties as required
must return all safe properties as required
"""
assert all(prop.is_required for prop in configuration_printer.properties())
configuration_printer.values = {"password": "pa55w0rd"}
assert all(not prop.is_required for prop in configuration_printer.properties())
def test_title(configuration_printer: ConfigurationPrinter) -> None:
"""

View File

@ -22,8 +22,13 @@ def test_package_update(database: SQLite, configuration: Configuration, package_
patch2 = PkgbuildPatch("key", "value")
local = Path("local")
glob_mock = mocker.patch("pathlib.Path.glob", return_value=[".git", ".gitignore"])
mocker.patch(
"pathlib.Path.is_file",
autospec=True,
side_effect=lambda p: True if p == Path(".gitignore") else False)
glob_mock = mocker.patch("pathlib.Path.glob", return_value=[Path(".git"), Path(".gitignore")])
rmtree_mock = mocker.patch("shutil.rmtree")
unlink_mock = mocker.patch("pathlib.Path.unlink")
fetch_mock = mocker.patch("ahriman.core.build_tools.sources.Sources.fetch")
patches_mock = mocker.patch("ahriman.core.database.SQLite.patches_get", return_value=[patch1, patch2])
patches_write_mock = mocker.patch("ahriman.models.pkgbuild_patch.PkgbuildPatch.write")
@ -33,9 +38,9 @@ def test_package_update(database: SQLite, configuration: Configuration, package_
glob_mock.assert_called_once_with(".git*")
rmtree_mock.assert_has_calls([
MockCall(local / package_ahriman.base, ignore_errors=True),
MockCall(local / package_ahriman.base / ".git"),
MockCall(local / package_ahriman.base / ".gitignore"),
MockCall(Path(".git")),
])
unlink_mock.assert_called_once_with()
fetch_mock.assert_called_once_with(pytest.helpers.anyvar(int), package_ahriman.remote)
patches_mock.assert_called_once_with(package_ahriman.base)
patches_write_mock.assert_has_calls([

View File

@ -120,7 +120,7 @@ def test_updates_local(update_handler: UpdateHandler, package_ahriman: Package,
assert update_handler.updates_local(vcs=True) == [package_ahriman]
fetch_mock.assert_called_once_with(Path(package_ahriman.base), remote=None)
package_load_mock.assert_called_once_with(Path(package_ahriman.base))
package_load_mock.assert_called_once_with(Path(package_ahriman.base), "x86_64")
status_client_mock.assert_called_once_with(package_ahriman.base)
package_is_outdated_mock.assert_called_once_with(
package_ahriman, update_handler.paths,

View File

@ -36,11 +36,25 @@ def test_process_error(spawner: Spawn) -> None:
assert spawner.queue.empty()
def test_spawn_process(spawner: Spawn, mocker: MockerFixture) -> None:
"""
must correctly spawn child process
"""
start_mock = mocker.patch("multiprocessing.Process.start")
spawner._spawn_process("add", "ahriman", now="", maybe="?")
start_mock.assert_called_once_with()
spawner.args_parser.parse_args.assert_called_once_with([
"--architecture", spawner.architecture, "--configuration", str(spawner.configuration.path),
"add", "ahriman", "--now", "--maybe", "?"
])
def test_key_import(spawner: Spawn, mocker: MockerFixture) -> None:
"""
must call key import
"""
spawn_mock = mocker.patch("ahriman.core.spawn.Spawn.spawn_process")
spawn_mock = mocker.patch("ahriman.core.spawn.Spawn._spawn_process")
spawner.key_import("0xdeadbeaf", None)
spawn_mock.assert_called_once_with("service-key-import", "0xdeadbeaf")
@ -49,7 +63,7 @@ def test_key_import_with_server(spawner: Spawn, mocker: MockerFixture) -> None:
"""
must call key import with server specified
"""
spawn_mock = mocker.patch("ahriman.core.spawn.Spawn.spawn_process")
spawn_mock = mocker.patch("ahriman.core.spawn.Spawn._spawn_process")
spawner.key_import("0xdeadbeaf", "keyserver.ubuntu.com")
spawn_mock.assert_called_once_with("service-key-import", "0xdeadbeaf", **{"key-server": "keyserver.ubuntu.com"})
@ -58,7 +72,7 @@ def test_packages_add(spawner: Spawn, mocker: MockerFixture) -> None:
"""
must call package addition
"""
spawn_mock = mocker.patch("ahriman.core.spawn.Spawn.spawn_process")
spawn_mock = mocker.patch("ahriman.core.spawn.Spawn._spawn_process")
spawner.packages_add(["ahriman", "linux"], now=False)
spawn_mock.assert_called_once_with("package-add", "ahriman", "linux", source="aur")
@ -67,7 +81,7 @@ def test_packages_add_with_build(spawner: Spawn, mocker: MockerFixture) -> None:
"""
must call package addition with update
"""
spawn_mock = mocker.patch("ahriman.core.spawn.Spawn.spawn_process")
spawn_mock = mocker.patch("ahriman.core.spawn.Spawn._spawn_process")
spawner.packages_add(["ahriman", "linux"], now=True)
spawn_mock.assert_called_once_with("package-add", "ahriman", "linux", source="aur", now="")
@ -76,7 +90,7 @@ def test_packages_rebuild(spawner: Spawn, mocker: MockerFixture) -> None:
"""
must call package rebuild
"""
spawn_mock = mocker.patch("ahriman.core.spawn.Spawn.spawn_process")
spawn_mock = mocker.patch("ahriman.core.spawn.Spawn._spawn_process")
spawner.packages_rebuild("python")
spawn_mock.assert_called_once_with("repo-rebuild", **{"depends-on": "python"})
@ -85,7 +99,7 @@ def test_packages_remove(spawner: Spawn, mocker: MockerFixture) -> None:
"""
must call package removal
"""
spawn_mock = mocker.patch("ahriman.core.spawn.Spawn.spawn_process")
spawn_mock = mocker.patch("ahriman.core.spawn.Spawn._spawn_process")
spawner.packages_remove(["ahriman", "linux"])
spawn_mock.assert_called_once_with("package-remove", "ahriman", "linux")
@ -94,25 +108,11 @@ def test_packages_update(spawner: Spawn, mocker: MockerFixture) -> None:
"""
must call repo update
"""
spawn_mock = mocker.patch("ahriman.core.spawn.Spawn.spawn_process")
spawn_mock = mocker.patch("ahriman.core.spawn.Spawn._spawn_process")
spawner.packages_update()
spawn_mock.assert_called_once_with("repo-update")
def test_spawn_process(spawner: Spawn, mocker: MockerFixture) -> None:
"""
must correctly spawn child process
"""
start_mock = mocker.patch("multiprocessing.Process.start")
spawner.spawn_process("add", "ahriman", now="", maybe="?")
start_mock.assert_called_once_with()
spawner.args_parser.parse_args.assert_called_once_with([
"--architecture", spawner.architecture, "--configuration", str(spawner.configuration.path),
"add", "ahriman", "--now", "--maybe", "?"
])
def test_run(spawner: Spawn, mocker: MockerFixture) -> None:
"""
must implement run method
@ -145,3 +145,14 @@ def test_run_pop(spawner: Spawn) -> None:
second.terminate.assert_called_once_with()
second.join.assert_called_once_with()
assert not spawner.active
def test_stop(spawner: Spawn) -> None:
"""
must gracefully terminate thread
"""
spawner.start()
spawner.stop()
spawner.join()
assert not spawner.is_alive()

View File

@ -358,6 +358,7 @@ def test_walk(resource_path_root: Path) -> None:
resource_path_root / "models" / "package_akonadi_aur",
resource_path_root / "models" / "package_ahriman_srcinfo",
resource_path_root / "models" / "package_gcc10_srcinfo",
resource_path_root / "models" / "package_jellyfin-ffmpeg5-bin_srcinfo",
resource_path_root / "models" / "package_tpacpi-bat-git_srcinfo",
resource_path_root / "models" / "package_yay_srcinfo",
resource_path_root / "web" / "templates" / "build-status" / "alerts.jinja2",

View File

@ -13,7 +13,6 @@ from ahriman.models.package import Package
from ahriman.models.package_description import PackageDescription
from ahriman.models.package_source import PackageSource
from ahriman.models.remote_source import RemoteSource
from ahriman.models.user_identity import UserIdentity
@pytest.fixture
@ -149,14 +148,3 @@ def pyalpm_package_description_ahriman(package_description_ahriman: PackageDescr
type(mock).provides = PropertyMock(return_value=package_description_ahriman.provides)
type(mock).url = PropertyMock(return_value=package_description_ahriman.url)
return mock
@pytest.fixture
def user_identity() -> UserIdentity:
"""
identity fixture
Returns:
UserIdentity: user identity test instance
"""
return UserIdentity("username", int(time.time()) + 30)

View File

@ -51,7 +51,7 @@ def test_depends_build_with_version_and_overlap(mocker: MockerFixture, resource_
srcinfo = (resource_path_root / "models" / "package_gcc10_srcinfo").read_text()
mocker.patch("ahriman.models.package.Package._check_output", return_value=srcinfo)
package_gcc10 = Package.from_build(Path("local"))
package_gcc10 = Package.from_build(Path("local"), "x86_64")
assert package_gcc10.depends_build == {"glibc", "doxygen", "binutils", "git", "libmpc", "python", "zstd"}
@ -125,6 +125,14 @@ def test_licenses(package_ahriman: Package) -> None:
assert sorted(package_ahriman.licenses) == package_ahriman.licenses
def test_packages_full(package_ahriman: Package) -> None:
"""
must return full list of packages including provides
"""
package_ahriman.packages[package_ahriman.base].provides = [f"{package_ahriman.base}-git"]
assert package_ahriman.packages_full == [package_ahriman.base, f"{package_ahriman.base}-git"]
def test_from_archive(package_ahriman: Package, pyalpm_handle: MagicMock, mocker: MockerFixture) -> None:
"""
must construct package from alpm library
@ -154,7 +162,7 @@ def test_from_build(package_ahriman: Package, mocker: MockerFixture, resource_pa
srcinfo = (resource_path_root / "models" / "package_ahriman_srcinfo").read_text()
mocker.patch("ahriman.models.package.Package._check_output", return_value=srcinfo)
package = Package.from_build(Path("path"))
package = Package.from_build(Path("path"), "x86_64")
assert package_ahriman.packages.keys() == package.packages.keys()
package_ahriman.packages = package.packages # we are not going to test PackageDescription here
package_ahriman.remote = None
@ -168,7 +176,7 @@ def test_from_build_multiple_packages(mocker: MockerFixture, resource_path_root:
srcinfo = (resource_path_root / "models" / "package_gcc10_srcinfo").read_text()
mocker.patch("ahriman.models.package.Package._check_output", return_value=srcinfo)
package = Package.from_build(Path("path"))
package = Package.from_build(Path("path"), "x86_64")
assert package.packages == {
"gcc10": PackageDescription(
depends=["gcc10-libs=10.3.0-2", "binutils>=2.28", "libmpc", "zstd"],
@ -188,6 +196,34 @@ def test_from_build_multiple_packages(mocker: MockerFixture, resource_path_root:
}
def test_from_build_architecture(mocker: MockerFixture, resource_path_root: Path) -> None:
"""
must construct package with architecture specific depends list
"""
srcinfo = (resource_path_root / "models" / "package_jellyfin-ffmpeg5-bin_srcinfo").read_text()
mocker.patch("ahriman.models.package.Package._check_output", return_value=srcinfo)
package = Package.from_build(Path("path"), "x86_64")
assert package.packages == {
"jellyfin-ffmpeg5-bin": PackageDescription(
depends=["glibc"],
make_depends=[],
opt_depends=[
"intel-media-driver: for Intel VAAPI support (Broadwell and newer)",
"intel-media-sdk: for Intel Quick Sync Video",
"onevpl-intel-gpu: for Intel Quick Sync Video (12th Gen and newer)",
"intel-compute-runtime: for Intel OpenCL runtime based Tonemapping",
"libva-intel-driver: for Intel legacy VAAPI support (10th Gen and older)",
"libva-mesa-driver: for AMD VAAPI support",
"nvidia-utils: for Nvidia NVDEC/NVENC support",
"opencl-amd: for AMD OpenCL runtime based Tonemapping",
"vulkan-radeon: for AMD RADV Vulkan support",
"vulkan-intel: for Intel ANV Vulkan support",
],
),
}
def test_from_build_failed(package_ahriman: Package, mocker: MockerFixture) -> None:
"""
must raise exception if there are errors during srcinfo load
@ -196,7 +232,7 @@ def test_from_build_failed(package_ahriman: Package, mocker: MockerFixture) -> N
mocker.patch("ahriman.models.package.parse_srcinfo", return_value=({"packages": {}}, ["an error"]))
with pytest.raises(PackageInfoError):
Package.from_build(Path("path"))
Package.from_build(Path("path"), "x86_64")
def test_from_json_view_1(package_ahriman: Package) -> None:

View File

@ -1,64 +0,0 @@
from ahriman.models.user_identity import UserIdentity
def test_from_identity(user_identity: UserIdentity) -> None:
"""
must construct identity object from string
"""
identity = UserIdentity.from_identity(f"{user_identity.username} {user_identity.expire_at}")
assert identity == user_identity
def test_from_identity_expired(user_identity: UserIdentity) -> None:
"""
must construct None from expired identity
"""
user_identity = UserIdentity(username=user_identity.username, expire_at=user_identity.expire_at - 60)
assert UserIdentity.from_identity(f"{user_identity.username} {user_identity.expire_at}") is None
def test_from_identity_no_split() -> None:
"""
must construct None from invalid string
"""
assert UserIdentity.from_identity("username") is None
def test_from_identity_not_int() -> None:
"""
must construct None from invalid timestamp
"""
assert UserIdentity.from_identity("username timestamp") is None
def test_from_username() -> None:
"""
must construct identity from username
"""
identity = UserIdentity.from_username("username", 0)
assert identity.username == "username"
# we want to check timestamp too, but later
def test_expire_when() -> None:
"""
must return correct expiration time
"""
assert UserIdentity.expire_when(-1) < UserIdentity.expire_when(0) < UserIdentity.expire_when(1)
def test_is_expired(user_identity: UserIdentity) -> None:
"""
must return expired flag for expired identities
"""
assert not user_identity.is_expired()
user_identity = UserIdentity(username=user_identity.username, expire_at=user_identity.expire_at - 60)
assert user_identity.is_expired()
def test_to_identity(user_identity: UserIdentity) -> None:
"""
must return correct identity string
"""
assert user_identity == UserIdentity.from_identity(user_identity.to_identity())

View File

@ -3,27 +3,15 @@ import socket
from aiohttp import web
from aiohttp.test_utils import TestClient
from cryptography import fernet
from pytest_mock import MockerFixture
from unittest.mock import AsyncMock
from unittest.mock import AsyncMock, call as MockCall
from ahriman.core.auth import Auth
from ahriman.core.configuration import Configuration
from ahriman.models.user import User
from ahriman.models.user_access import UserAccess
from ahriman.models.user_identity import UserIdentity
from ahriman.web.middlewares.auth_handler import auth_handler, AuthorizationPolicy, setup_auth
def _identity(username: str) -> str:
"""
generate identity from user
Args:
username(str): name of the user
Returns:
str: user identity string
"""
return f"{username} {UserIdentity.expire_when(60)}"
from ahriman.web.middlewares.auth_handler import AuthorizationPolicy, auth_handler, cookie_secret_key, setup_auth
async def test_authorized_userid(authorization_policy: AuthorizationPolicy, user: User, mocker: MockerFixture) -> None:
@ -31,14 +19,14 @@ async def test_authorized_userid(authorization_policy: AuthorizationPolicy, user
must return authorized user id
"""
mocker.patch("ahriman.core.database.SQLite.user_get", return_value=user)
assert await authorization_policy.authorized_userid(_identity(user.username)) == user.username
assert await authorization_policy.authorized_userid(user.username) == user.username
async def test_authorized_userid_unknown(authorization_policy: AuthorizationPolicy, user: User) -> None:
"""
must not allow unknown user id for authorization
"""
assert await authorization_policy.authorized_userid(_identity("somerandomname")) is None
assert await authorization_policy.authorized_userid("somerandomname") is None
assert await authorization_policy.authorized_userid("somerandomname") is None
@ -49,11 +37,13 @@ async def test_permits(authorization_policy: AuthorizationPolicy, user: User) ->
authorization_policy.validator = AsyncMock()
authorization_policy.validator.verify_access.side_effect = lambda username, *args: username == user.username
assert await authorization_policy.permits(_identity(user.username), user.access, "/endpoint")
authorization_policy.validator.verify_access.assert_called_once_with(user.username, user.access, "/endpoint")
assert await authorization_policy.permits(user.username, user.access, "/endpoint")
assert not await authorization_policy.permits("somerandomname", user.access, "/endpoint")
assert not await authorization_policy.permits(_identity("somerandomname"), user.access, "/endpoint")
assert not await authorization_policy.permits(user.username, user.access, "/endpoint")
authorization_policy.validator.verify_access.assert_has_calls([
MockCall(user.username, user.access, "/endpoint"),
MockCall("somerandomname", user.access, "/endpoint"),
])
async def test_auth_handler_unix_socket(client_with_auth: TestClient, mocker: MockerFixture) -> None:
@ -175,11 +165,28 @@ async def test_auth_handler_write(mocker: MockerFixture) -> None:
check_permission_mock.assert_called_once_with(aiohttp_request, UserAccess.Full, aiohttp_request.path)
def test_setup_auth(application_with_auth: web.Application, auth: Auth, mocker: MockerFixture) -> None:
def test_cookie_secret_key(configuration: Configuration) -> None:
"""
must generate fernet key
"""
secret_key = cookie_secret_key(configuration)
assert isinstance(secret_key, fernet.Fernet)
def test_cookie_secret_key_cached(configuration: Configuration) -> None:
"""
must use cookie key as set by configuration
"""
configuration.set_option("auth", "cookie_secret_key", fernet.Fernet.generate_key().decode("utf8"))
assert cookie_secret_key(configuration) is not None
def test_setup_auth(application_with_auth: web.Application, configuration: Configuration, auth: Auth,
mocker: MockerFixture) -> None:
"""
must set up authorization
"""
setup_mock = mocker.patch("aiohttp_security.setup")
application = setup_auth(application_with_auth, auth)
application = setup_auth(application_with_auth, configuration, auth)
assert application.get("validator") is not None
setup_mock.assert_called_once_with(application_with_auth, pytest.helpers.anyvar(int), pytest.helpers.anyvar(int))

View File

@ -100,7 +100,7 @@ def test_run(application: web.Application, mocker: MockerFixture) -> None:
run_server(application)
run_application_mock.assert_called_once_with(
application, host="127.0.0.1", port=port, sock=None, handle_signals=False,
application, host="127.0.0.1", port=port, sock=None, handle_signals=True,
access_log=pytest.helpers.anyvar(int), access_log_class=FilteredAccessLogger
)
@ -115,7 +115,7 @@ def test_run_with_auth(application_with_auth: web.Application, mocker: MockerFix
run_server(application_with_auth)
run_application_mock.assert_called_once_with(
application_with_auth, host="127.0.0.1", port=port, sock=None, handle_signals=False,
application_with_auth, host="127.0.0.1", port=port, sock=None, handle_signals=True,
access_log=pytest.helpers.anyvar(int), access_log_class=FilteredAccessLogger
)
@ -130,7 +130,7 @@ def test_run_with_debug(application_with_debug: web.Application, mocker: MockerF
run_server(application_with_debug)
run_application_mock.assert_called_once_with(
application_with_debug, host="127.0.0.1", port=port, sock=None, handle_signals=False,
application_with_debug, host="127.0.0.1", port=port, sock=None, handle_signals=True,
access_log=pytest.helpers.anyvar(int), access_log_class=FilteredAccessLogger
)
@ -147,6 +147,6 @@ def test_run_with_socket(application: web.Application, mocker: MockerFixture) ->
run_server(application)
socket_mock.assert_called_once_with(application["configuration"], application)
run_application_mock.assert_called_once_with(
application, host="127.0.0.1", port=port, sock=42, handle_signals=False,
application, host="127.0.0.1", port=port, sock=42, handle_signals=True,
access_log=pytest.helpers.anyvar(int), access_log_class=FilteredAccessLogger
)

View File

@ -0,0 +1,28 @@
pkgbase = jellyfin-ffmpeg5-bin
pkgdesc = FFmpeg5 binary version for Jellyfin
pkgver = 5.1.2
pkgrel = 7
url = https://github.com/jellyfin/jellyfin-ffmpeg
arch = x86_64
arch = aarch64
license = GPL3
optdepends = intel-media-driver: for Intel VAAPI support (Broadwell and newer)
optdepends = intel-media-sdk: for Intel Quick Sync Video
optdepends = onevpl-intel-gpu: for Intel Quick Sync Video (12th Gen and newer)
optdepends = intel-compute-runtime: for Intel OpenCL runtime based Tonemapping
optdepends = libva-intel-driver: for Intel legacy VAAPI support (10th Gen and older)
optdepends = libva-mesa-driver: for AMD VAAPI support
optdepends = nvidia-utils: for Nvidia NVDEC/NVENC support
optdepends = opencl-amd: for AMD OpenCL runtime based Tonemapping
optdepends = vulkan-radeon: for AMD RADV Vulkan support
optdepends = vulkan-intel: for Intel ANV Vulkan support
conflicts = jellyfin-ffmpeg
conflicts = jellyfin-ffmpeg5
source_x86_64 = https://repo.jellyfin.org/releases/ffmpeg/5.1.2-7/jellyfin-ffmpeg_5.1.2-7_portable_linux64-gpl.tar.xz
depends_x86_64 = glibc>=2.23
sha256sums_x86_64 = 78420fd1edbaf24a07e92938878d8582d895e009cae02c8e9d5be3f26de905e3
source_aarch64 = https://repo.jellyfin.org/releases/ffmpeg/5.1.2-7/jellyfin-ffmpeg_5.1.2-7_portable_linuxarm64-gpl.tar.xz
depends_aarch64 = glibc>=2.27
sha256sums_aarch64 = 8ac4066981f203c2b442754eaf7286b4e481df9692d0ff8910a824d89c831df0
pkgname = jellyfin-ffmpeg5-bin