Compare commits

...

17 Commits

Author SHA1 Message Date
bd0f850d25 Release 2.10.1 2023-07-28 03:21:19 +03:00
5904727da2 do not create application in shell handler
The main reason for having shell handler is to be able to fix if
something (e.g. migrations) goes wrong. In this way we need to reduce
actions inside this wrapper
2023-07-28 03:06:28 +03:00
263c53bac5 rase 405 error in case if GET login method is used whereas no aioauth
library installed
2023-07-23 03:01:25 +03:00
6743f1d62a optimize schemas import 2023-07-23 02:24:39 +03:00
931ff9bd98 Release 2.10.0 2023-07-22 05:28:57 +03:00
6b3fc3a6a0 add support of table filter controls (#101) 2023-07-21 02:10:26 +03:00
b7852f55c8 remove unsafe flag from handlers
This flag became reduntant there and tree creation has been moved to
lock
2023-07-07 03:25:05 +03:00
721b447767 fix code block in docs 2023-07-06 19:17:11 +03:00
b80ea80e9d add salt generator to setup command instead 2023-07-06 19:16:49 +03:00
8e9da5baab register dependency package before build
If package has been added as dependency, the service miss remote as well
as causes some 400 errors in reporter
2023-07-06 03:11:19 +03:00
a443abb94e handle packages load from aur by package name also
In general package names array may not contain package base, thus it
leads to inability to load packages from aur by its base during update
process
2023-07-01 15:55:04 +03:00
61c565ab0d explicitly pass user agent for the arch linux sites requests 2023-06-26 02:52:08 +03:00
10100b20e1 print configuration paths in dump command 2023-06-11 15:11:34 +03:00
2922bb9d72 remove salt generation from users handler
It causes issues, because users handler is operating with service user,
but writtinng salt requires root privileges
2023-06-05 05:25:10 +03:00
17f5f41e36 complitely disable signature check for local repository in devtools
It appears that with optional level pacman still tries to validate the
key, which can lead to errors whille processing in docker container
2023-06-05 04:41:03 +03:00
c22ddd71d9 packager documentation update 2023-06-05 03:40:02 +03:00
4b984afb64 packagers support (#100) 2023-06-05 02:37:19 +03:00
182 changed files with 6156 additions and 5149 deletions

View File

@ -16,7 +16,7 @@ Wrapper for managing custom repository inspired by [repo-scripts](https://github
* VCS packages support.
* Official repository support.
* Ability to patch AUR packages and even create package from local PKGBUILDs.
* Sign support with gpg (repository, package, per package settings).
* Sign support with gpg (repository, package), multiple packagers support.
* Triggers for repository updates, e.g. synchronization to remote services (rsync, s3 and github) and report generation (email, html, telegram).
* Repository status interface with optional authorization and control options:

View File

@ -58,7 +58,7 @@ systemd-machine-id-setup &> /dev/null
# otherwise we prepend executable by sudo command
if [ -n "$AHRIMAN_FORCE_ROOT" ]; then
AHRIMAN_EXECUTABLE=("ahriman")
elif ahriman help-commands-unsafe --command="$*" &> /dev/null; then
elif ahriman help-commands-unsafe -- "$@" &> /dev/null; then
AHRIMAN_EXECUTABLE=("sudo" "-u" "$AHRIMAN_USER" "--" "ahriman")
else
AHRIMAN_EXECUTABLE=("ahriman")

File diff suppressed because it is too large Load Diff

Before

Width:  |  Height:  |  Size: 762 KiB

After

Width:  |  Height:  |  Size: 809 KiB

View File

@ -1,4 +1,4 @@
.TH AHRIMAN "1" "2023\-05\-28" "ahriman" "Generated Python Manual"
.TH AHRIMAN "1" "2023\-07\-28" "ahriman" "Generated Python Manual"
.SH NAME
ahriman
.SH SYNOPSIS
@ -199,13 +199,12 @@ show help message for application or command and exit
show help message for specific command
.SH COMMAND \fI\,'ahriman help\-commands\-unsafe'\/\fR
usage: ahriman help\-commands\-unsafe [\-h] [\-\-command COMMAND]
usage: ahriman help\-commands\-unsafe [\-h] [command ...]
list unsafe commands as defined in default args
.SH OPTIONS \fI\,'ahriman help\-commands\-unsafe'\/\fR
.TP
\fB\-\-command\fR \fI\,COMMAND\/\fR
\fBcommand\fR
instead of showing commands, just test command line for unsafe subcommand and return 0 in case if command is safe and 1
otherwise
@ -226,7 +225,7 @@ print application and its dependencies versions
.SH COMMAND \fI\,'ahriman package\-add'\/\fR
usage: ahriman package\-add [\-h] [\-\-dependencies | \-\-no\-dependencies] [\-e] [\-n] [\-y]
[\-s {auto,archive,aur,directory,local,remote,repository}]
[\-s {auto,archive,aur,directory,local,remote,repository}] [\-u USERNAME]
package [package ...]
add existing or new package to the build queue
@ -256,6 +255,10 @@ download fresh package databases from the mirror before actions, \-yy to force r
\fB\-s\fR \fI\,{auto,archive,aur,directory,local,remote,repository}\/\fR, \fB\-\-source\fR \fI\,{auto,archive,aur,directory,local,remote,repository}\/\fR
explicitly specify the package source for this command
.TP
\fB\-u\fR \fI\,USERNAME\/\fR, \fB\-\-username\fR \fI\,USERNAME\/\fR
build as user
.SH COMMAND \fI\,'ahriman package\-remove'\/\fR
usage: ahriman package\-remove [\-h] package [package ...]
@ -457,7 +460,7 @@ download fresh package databases from the mirror before actions, \-yy to force r
.SH COMMAND \fI\,'ahriman repo\-rebuild'\/\fR
usage: ahriman repo\-rebuild [\-h] [\-\-depends\-on DEPENDS_ON] [\-\-dry\-run] [\-\-from\-database] [\-e]
[\-s {unknown,pending,building,failed,success}]
[\-s {unknown,pending,building,failed,success}] [\-u USERNAME]
force rebuild whole repository
@ -484,6 +487,10 @@ return non\-zero exit status if result is empty
\fB\-s\fR \fI\,{unknown,pending,building,failed,success}\/\fR, \fB\-\-status\fR \fI\,{unknown,pending,building,failed,success}\/\fR
filter packages by status. Requires \-\-from\-database to be set
.TP
\fB\-u\fR \fI\,USERNAME\/\fR, \fB\-\-username\fR \fI\,USERNAME\/\fR
build as user
.SH COMMAND \fI\,'ahriman repo\-remove\-unknown'\/\fR
usage: ahriman repo\-remove\-unknown [\-h] [\-\-dry\-run]
@ -553,7 +560,7 @@ instead of running all triggers as set by configuration, just process specified
.SH COMMAND \fI\,'ahriman repo\-update'\/\fR
usage: ahriman repo\-update [\-h] [\-\-aur | \-\-no\-aur] [\-\-dependencies | \-\-no\-dependencies] [\-\-dry\-run] [\-e]
[\-\-local | \-\-no\-local] [\-\-manual | \-\-no\-manual] [\-\-vcs | \-\-no\-vcs] [\-y]
[\-\-local | \-\-no\-local] [\-\-manual | \-\-no\-manual] [\-u USERNAME] [\-\-vcs | \-\-no\-vcs] [\-y]
[package ...]
check for packages updates and run build process if requested
@ -587,6 +594,10 @@ enable or disable checking of local packages for updates
\fB\-\-manual\fR, \fB\-\-no\-manual\fR
include or exclude manual updates
.TP
\fB\-u\fR \fI\,USERNAME\/\fR, \fB\-\-username\fR \fI\,USERNAME\/\fR
build as user
.TP
\fB\-\-vcs\fR, \fB\-\-no\-vcs\fR
fetch actual version of VCS packages
@ -658,9 +669,10 @@ key server for key import
.SH COMMAND \fI\,'ahriman service\-setup'\/\fR
usage: ahriman service\-setup [\-h] [\-\-build\-as\-user BUILD_AS_USER] [\-\-build\-command BUILD_COMMAND]
[\-\-from\-configuration FROM_CONFIGURATION] [\-\-makeflags\-jobs | \-\-no\-makeflags\-jobs]
[\-\-mirror MIRROR] [\-\-multilib | \-\-no\-multilib] \-\-packager PACKAGER \-\-repository REPOSITORY
[\-\-sign\-key SIGN_KEY] [\-\-sign\-target {disabled,packages,repository}] [\-\-web\-port WEB_PORT]
[\-\-from\-configuration FROM_CONFIGURATION] [\-\-generate\-salt | \-\-no\-generate\-salt]
[\-\-makeflags\-jobs | \-\-no\-makeflags\-jobs] [\-\-mirror MIRROR] [\-\-multilib | \-\-no\-multilib]
\-\-packager PACKAGER \-\-repository REPOSITORY [\-\-sign\-key SIGN_KEY]
[\-\-sign\-target {disabled,packages,repository}] [\-\-web\-port WEB_PORT]
[\-\-web\-unix\-socket WEB_UNIX_SOCKET]
create initial service configuration, requires root
@ -678,6 +690,10 @@ build command prefix
\fB\-\-from\-configuration\fR \fI\,FROM_CONFIGURATION\/\fR
path to default devtools pacman configuration
.TP
\fB\-\-generate\-salt\fR, \fB\-\-no\-generate\-salt\fR
generate salt for user passwords
.TP
\fB\-\-makeflags\-jobs\fR, \fB\-\-no\-makeflags\-jobs\fR
append MAKEFLAGS variable with parallelism set to number of cores
@ -717,14 +733,15 @@ path to unix socket used for interprocess communications
.SH COMMAND \fI\,'ahriman service\-shell'\/\fR
usage: ahriman service\-shell [\-h] [code]
drop into python shell while having created application
drop into python shell
.TP
\fBcode\fR
instead of dropping into shell, just execute the specified code
.SH COMMAND \fI\,'ahriman user\-add'\/\fR
usage: ahriman user\-add [\-h] [\-p PASSWORD] [\-r {unauthorized,read,reporter,full}] [\-s] username
usage: ahriman user\-add [\-h] [\-\-key KEY] [\-\-packager PACKAGER] [\-p PASSWORD] [\-r {unauthorized,read,reporter,full}]
username
update user for web services with the given password and role. In case if password was not entered it will be asked interactively
@ -733,6 +750,14 @@ update user for web services with the given password and role. In case if passwo
username for web service
.SH OPTIONS \fI\,'ahriman user\-add'\/\fR
.TP
\fB\-\-key\fR \fI\,KEY\/\fR
optional PGP key used by this user. The private key must be imported
.TP
\fB\-\-packager\fR \fI\,PACKAGER\/\fR
optional packager id used for build process in form of `Name Surname <mail@example.com>`
.TP
\fB\-p\fR \fI\,PASSWORD\/\fR, \fB\-\-password\fR \fI\,PASSWORD\/\fR
user password. Blank password will be treated as empty password, which is in particular must be used for OAuth2
@ -742,10 +767,6 @@ authorization type.
\fB\-r\fR \fI\,{unauthorized,read,reporter,full}\/\fR, \fB\-\-role\fR \fI\,{unauthorized,read,reporter,full}\/\fR
user access level
.TP
\fB\-s\fR, \fB\-\-secure\fR
set file permissions to user\-only
.SH COMMAND \fI\,'ahriman user\-list'\/\fR
usage: ahriman user\-list [\-h] [\-e] [\-r {unauthorized,read,reporter,full}] [username]

View File

@ -68,6 +68,14 @@ ahriman.core.database.migrations.m007\_check\_depends module
:no-undoc-members:
:show-inheritance:
ahriman.core.database.migrations.m008\_packagers module
-------------------------------------------------------
.. automodule:: ahriman.core.database.migrations.m008_packagers
:members:
:no-undoc-members:
:show-inheritance:
Module contents
---------------

View File

@ -20,6 +20,14 @@ ahriman.core.formatters.build\_printer module
:no-undoc-members:
:show-inheritance:
ahriman.core.formatters.configuration\_paths\_printer module
------------------------------------------------------------
.. automodule:: ahriman.core.formatters.configuration_paths_printer
:members:
:no-undoc-members:
:show-inheritance:
ahriman.core.formatters.configuration\_printer module
-----------------------------------------------------

View File

@ -116,6 +116,14 @@ ahriman.models.package\_source module
:no-undoc-members:
:show-inheritance:
ahriman.models.packagers module
-------------------------------
.. automodule:: ahriman.models.packagers
:members:
:no-undoc-members:
:show-inheritance:
ahriman.models.pacman\_synchronization module
---------------------------------------------

View File

@ -33,6 +33,7 @@ This package contains everything required for the most of application actions an
* ``ahriman.core.alpm`` package controls pacman related functions. It provides wrappers for ``pyalpm`` library and safe calls for repository tools (``repo-add`` and ``repo-remove``). Also this package contains ``ahriman.core.alpm.remote`` package which provides wrapper for remote sources (e.g. AUR RPC and official repositories RPC).
* ``ahriman.core.auth`` package provides classes for authorization methods used by web mostly. Base class is ``ahriman.core.auth.Auth`` which must be called by ``load`` method.
* ``ahriman.core.build_tools`` is a package which provides wrapper for ``devtools`` commands.
* ``ahriman.core.configuration`` contains extension for standard ``configparser`` library and some validation related classes.
* ``ahriman.core.database`` is everything including data and schema migrations for database.
* ``ahriman.core.formatters`` package provides ``Printer`` sub-classes for printing data (e.g. package properties) to stdout which are used by some handlers.
* ``ahriman.core.gitremote`` is a package with remote PKGBUILD triggers. Should not be called directly.
@ -41,12 +42,12 @@ This package contains everything required for the most of application actions an
* ``ahriman.core.repository`` contains several traits and base repository (``ahriman.core.repository.Repository`` class) implementation.
* ``ahriman.core.sign`` package provides sign feature (only gpg calls are available).
* ``ahriman.core.status`` contains helpers and watcher class which are required for web application. Reporter must be initialized by using ``ahriman.core.status.client.Client.load`` method.
* ``ahriman.core.support`` provides plugins for support packages (mirrorlist and keyring) generation.
* ``ahriman.core.triggers`` package contains base trigger classes. Classes from this package must be imported in order to implement user extensions. In fact, ``ahriman.core.report`` and ``ahriman.core.upload`` use this package.
* ``ahriman.core.upload`` package provides sync feature, should not be called directly.
This package also provides some generic functions and classes which may be used by other packages:
* ``ahriman.core.configuration.Configuration`` is an extension for standard ``configparser`` library.
* ``ahriman.core.exceptions`` provides custom exceptions.
* ``ahriman.core.spawn.Spawn`` is a tool which can spawn another ``ahriman`` process. This feature is used by web application.
* ``ahriman.core.tree`` is a dependency tree implementation.
@ -62,20 +63,23 @@ It provides models for any other part of application. Unlike ``ahriman.core`` pa
Web application. It is important that this package is isolated from any other to allow it to be optional feature (i.e. dependencies which are required by the package are optional).
* ``ahriman.web.middlewares`` provides middlewares for request handlers.
* ``ahriman.web.schemas`` provides schemas (actually copy paste from dataclasses) used by swagger documentation.
* ``ahriman.web.views`` contains web views derived from aiohttp view class.
* ``ahriman.web.apispec`` provides generators for swagger documentation.
* ``ahriman.web.cors`` contains helpers for cross origin resource sharing middlewares.
* ``ahriman.web.routes`` creates routes for web application.
* ``ahriman.web.web`` provides main web application functions (e.g. start, initialization).
Application run
---------------
* Parse command line arguments, find command and related handler which is set by parser.
* Call ``Handler.execute`` method.
* Define list of architectures to run. In case if there is more than one architecture specified run several subprocesses or process in current process otherwise. Class attribute ``ALLOW_MULTI_ARCHITECTURE_RUN`` controls whether application can be run in multiple processes or not - this feature is required for some handlers (e.g. ``Web``) which should be able to spawn child process in daemon mode (it is impossible to do from daemonic processes).
* In each child process call lock functions.
* After success checks pass control to ``Handler.run`` method defined by specific handler class.
* Return result (success or failure) of each subprocess and exit from application.
* Some handlers may override their status and throw ``ExitCode`` exception. This exception is just silently suppressed and changes application exit code to ``1``.
#. Parse command line arguments, find command and related handler which is set by parser.
#. Call ``Handler.execute`` method.
#. Define list of architectures to run. In case if there is more than one architecture specified run several subprocesses or process in current process otherwise. Class attribute ``ALLOW_MULTI_ARCHITECTURE_RUN`` controls whether application can be run in multiple processes or not - this feature is required for some handlers (e.g. ``Web``) which should be able to spawn child process in daemon mode (it is impossible to do from daemonic processes).
#. In each child process call lock functions.
#. After success checks pass control to ``Handler.run`` method defined by specific handler class.
#. Return result (success or failure) of each subprocess and exit from application.
#. Some handlers may override their status and throw ``ExitCode`` exception. This exception is just silently suppressed and changes application exit code to ``1``.
In the most cases handlers spawn god class ``ahriman.application.application.Application`` class and call required methods.
@ -114,7 +118,7 @@ Schema and data migrations
The schema migration are applied according to current ``pragma user_info`` values, located at ``ahriman.core.database.migrations`` package and named as ``m000_migration_name.py`` (the preceding ``m`` is required in order to import migration content for tests). Additional class ``ahriman.core.database.migrations.Migrations`` reads all migrations automatically and applies them in alphabetical order.
These migrations also contain data migrations. Though the recommended way is to migrate data directly from SQL requests, sometimes it is required to have external data (like packages list) in order to set correct data. To do so, special method `migrate_data` is used.
These migrations can also contain data migrations. Though the recommended way is to migrate data directly from SQL requests, sometimes it is required to have external data (like packages list) in order to set correct data. To do so, special method `migrate_data` is used.
Type conversions
^^^^^^^^^^^^^^^^
@ -126,6 +130,12 @@ By default, it parses rows into python dictionary. In addition, the following ps
Basic flows
-----------
By default package build operations are performed with ``PACKAGER`` which is specified in ``makepkg.conf``, however, it is possible to override this variable from command line; in this case service performs lookup in the following way:
* If packager is not set, it reads environment variables (e.g. ``SUDO_USER`` and ``USER``), otherwise it uses value from command line.
* It checks users for the specified username and tries to extract packager variable from it.
* If packager value has been found, it will be passed as ``PACKAGER`` system variable (sudo configuration required).
Add new packages or rebuild existing
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -140,7 +150,7 @@ This logic can be overwritten by specifying the ``source`` parameter, which is p
Rebuild packages
^^^^^^^^^^^^^^^^
Same as add function for every package in repository. Optional filter by reverse dependency can be supplied.
Same as add function for every package in repository. Optional filters by reverse dependency or build status can be supplied.
Remove packages
^^^^^^^^^^^^^^^
@ -224,7 +234,7 @@ OAuth provider uses library definitions (``aioauth-client``) in order *authentic
OAuth's implementation also allows authenticating users via username + password (in the same way as mapping does) though it is not recommended for end-users and password must be left blank. In particular this feature can be used by service reporting (aka robots).
In addition, web service checks the source socket used. In case if it belongs to ``socket.AF_UNIX`` family, it will skip any furher checks considering the request to be performed in safe environment (e.g. on the same physical machine). This feature, in particular is being used by the reporter instances in case if socket address is set in configuration.
In addition, web service checks the source socket used. In case if it belongs to ``socket.AF_UNIX`` family, it will skip any further checks considering the request to be performed in safe environment (e.g. on the same physical machine). This feature, in particular is being used by the reporter instances in case if socket address is set in configuration.
In order to configure users there are special commands.
@ -319,3 +329,5 @@ External calls
^^^^^^^^^^^^^^
Web application provides external calls to control main service. It spawns child process with specific arguments and waits for its termination. This feature must be used either with authorization or in safe (i.e. when status page is not available world-wide) environment.
For most actions it also extracts user from authentication (if provided) and passes it to underlying process.

View File

@ -6,13 +6,13 @@ _shtab_ahriman_option_strings=('-h' '--help' '-a' '--architecture' '-c' '--confi
_shtab_ahriman_aur_search_option_strings=('-h' '--help' '-e' '--exit-code' '--info' '--no-info' '--sort-by')
_shtab_ahriman_search_option_strings=('-h' '--help' '-e' '--exit-code' '--info' '--no-info' '--sort-by')
_shtab_ahriman_help_option_strings=('-h' '--help')
_shtab_ahriman_help_commands_unsafe_option_strings=('-h' '--help' '--command')
_shtab_ahriman_help_commands_unsafe_option_strings=('-h' '--help')
_shtab_ahriman_help_updates_option_strings=('-h' '--help' '-e' '--exit-code')
_shtab_ahriman_help_version_option_strings=('-h' '--help')
_shtab_ahriman_version_option_strings=('-h' '--help')
_shtab_ahriman_package_add_option_strings=('-h' '--help' '--dependencies' '--no-dependencies' '-e' '--exit-code' '-n' '--now' '-y' '--refresh' '-s' '--source')
_shtab_ahriman_add_option_strings=('-h' '--help' '--dependencies' '--no-dependencies' '-e' '--exit-code' '-n' '--now' '-y' '--refresh' '-s' '--source')
_shtab_ahriman_package_update_option_strings=('-h' '--help' '--dependencies' '--no-dependencies' '-e' '--exit-code' '-n' '--now' '-y' '--refresh' '-s' '--source')
_shtab_ahriman_package_add_option_strings=('-h' '--help' '--dependencies' '--no-dependencies' '-e' '--exit-code' '-n' '--now' '-y' '--refresh' '-s' '--source' '-u' '--username')
_shtab_ahriman_add_option_strings=('-h' '--help' '--dependencies' '--no-dependencies' '-e' '--exit-code' '-n' '--now' '-y' '--refresh' '-s' '--source' '-u' '--username')
_shtab_ahriman_package_update_option_strings=('-h' '--help' '--dependencies' '--no-dependencies' '-e' '--exit-code' '-n' '--now' '-y' '--refresh' '-s' '--source' '-u' '--username')
_shtab_ahriman_package_remove_option_strings=('-h' '--help')
_shtab_ahriman_remove_option_strings=('-h' '--help')
_shtab_ahriman_package_status_option_strings=('-h' '--help' '--ahriman' '-e' '--exit-code' '--info' '--no-info' '-s' '--status')
@ -31,8 +31,8 @@ _shtab_ahriman_repo_create_keyring_option_strings=('-h' '--help')
_shtab_ahriman_repo_create_mirrorlist_option_strings=('-h' '--help')
_shtab_ahriman_repo_daemon_option_strings=('-h' '--help' '-i' '--interval' '--aur' '--no-aur' '--dependencies' '--no-dependencies' '--local' '--no-local' '--manual' '--no-manual' '--vcs' '--no-vcs' '-y' '--refresh')
_shtab_ahriman_daemon_option_strings=('-h' '--help' '-i' '--interval' '--aur' '--no-aur' '--dependencies' '--no-dependencies' '--local' '--no-local' '--manual' '--no-manual' '--vcs' '--no-vcs' '-y' '--refresh')
_shtab_ahriman_repo_rebuild_option_strings=('-h' '--help' '--depends-on' '--dry-run' '--from-database' '-e' '--exit-code' '-s' '--status')
_shtab_ahriman_rebuild_option_strings=('-h' '--help' '--depends-on' '--dry-run' '--from-database' '-e' '--exit-code' '-s' '--status')
_shtab_ahriman_repo_rebuild_option_strings=('-h' '--help' '--depends-on' '--dry-run' '--from-database' '-e' '--exit-code' '-s' '--status' '-u' '--username')
_shtab_ahriman_rebuild_option_strings=('-h' '--help' '--depends-on' '--dry-run' '--from-database' '-e' '--exit-code' '-s' '--status' '-u' '--username')
_shtab_ahriman_repo_remove_unknown_option_strings=('-h' '--help' '--dry-run')
_shtab_ahriman_remove_unknown_option_strings=('-h' '--help' '--dry-run')
_shtab_ahriman_repo_report_option_strings=('-h' '--help')
@ -45,8 +45,8 @@ _shtab_ahriman_repo_sync_option_strings=('-h' '--help')
_shtab_ahriman_sync_option_strings=('-h' '--help')
_shtab_ahriman_repo_tree_option_strings=('-h' '--help')
_shtab_ahriman_repo_triggers_option_strings=('-h' '--help')
_shtab_ahriman_repo_update_option_strings=('-h' '--help' '--aur' '--no-aur' '--dependencies' '--no-dependencies' '--dry-run' '-e' '--exit-code' '--local' '--no-local' '--manual' '--no-manual' '--vcs' '--no-vcs' '-y' '--refresh')
_shtab_ahriman_update_option_strings=('-h' '--help' '--aur' '--no-aur' '--dependencies' '--no-dependencies' '--dry-run' '-e' '--exit-code' '--local' '--no-local' '--manual' '--no-manual' '--vcs' '--no-vcs' '-y' '--refresh')
_shtab_ahriman_repo_update_option_strings=('-h' '--help' '--aur' '--no-aur' '--dependencies' '--no-dependencies' '--dry-run' '-e' '--exit-code' '--local' '--no-local' '--manual' '--no-manual' '-u' '--username' '--vcs' '--no-vcs' '-y' '--refresh')
_shtab_ahriman_update_option_strings=('-h' '--help' '--aur' '--no-aur' '--dependencies' '--no-dependencies' '--dry-run' '-e' '--exit-code' '--local' '--no-local' '--manual' '--no-manual' '-u' '--username' '--vcs' '--no-vcs' '-y' '--refresh')
_shtab_ahriman_service_clean_option_strings=('-h' '--help' '--cache' '--no-cache' '--chroot' '--no-chroot' '--manual' '--no-manual' '--packages' '--no-packages' '--pacman' '--no-pacman')
_shtab_ahriman_clean_option_strings=('-h' '--help' '--cache' '--no-cache' '--chroot' '--no-chroot' '--manual' '--no-manual' '--packages' '--no-packages' '--pacman' '--no-pacman')
_shtab_ahriman_repo_clean_option_strings=('-h' '--help' '--cache' '--no-cache' '--chroot' '--no-chroot' '--manual' '--no-manual' '--packages' '--no-packages' '--pacman' '--no-pacman')
@ -58,14 +58,14 @@ _shtab_ahriman_config_validate_option_strings=('-h' '--help' '-e' '--exit-code')
_shtab_ahriman_repo_config_validate_option_strings=('-h' '--help' '-e' '--exit-code')
_shtab_ahriman_service_key_import_option_strings=('-h' '--help' '--key-server')
_shtab_ahriman_key_import_option_strings=('-h' '--help' '--key-server')
_shtab_ahriman_service_setup_option_strings=('-h' '--help' '--build-as-user' '--build-command' '--from-configuration' '--makeflags-jobs' '--no-makeflags-jobs' '--mirror' '--multilib' '--no-multilib' '--packager' '--repository' '--sign-key' '--sign-target' '--web-port' '--web-unix-socket')
_shtab_ahriman_init_option_strings=('-h' '--help' '--build-as-user' '--build-command' '--from-configuration' '--makeflags-jobs' '--no-makeflags-jobs' '--mirror' '--multilib' '--no-multilib' '--packager' '--repository' '--sign-key' '--sign-target' '--web-port' '--web-unix-socket')
_shtab_ahriman_repo_init_option_strings=('-h' '--help' '--build-as-user' '--build-command' '--from-configuration' '--makeflags-jobs' '--no-makeflags-jobs' '--mirror' '--multilib' '--no-multilib' '--packager' '--repository' '--sign-key' '--sign-target' '--web-port' '--web-unix-socket')
_shtab_ahriman_repo_setup_option_strings=('-h' '--help' '--build-as-user' '--build-command' '--from-configuration' '--makeflags-jobs' '--no-makeflags-jobs' '--mirror' '--multilib' '--no-multilib' '--packager' '--repository' '--sign-key' '--sign-target' '--web-port' '--web-unix-socket')
_shtab_ahriman_setup_option_strings=('-h' '--help' '--build-as-user' '--build-command' '--from-configuration' '--makeflags-jobs' '--no-makeflags-jobs' '--mirror' '--multilib' '--no-multilib' '--packager' '--repository' '--sign-key' '--sign-target' '--web-port' '--web-unix-socket')
_shtab_ahriman_service_setup_option_strings=('-h' '--help' '--build-as-user' '--build-command' '--from-configuration' '--generate-salt' '--no-generate-salt' '--makeflags-jobs' '--no-makeflags-jobs' '--mirror' '--multilib' '--no-multilib' '--packager' '--repository' '--sign-key' '--sign-target' '--web-port' '--web-unix-socket')
_shtab_ahriman_init_option_strings=('-h' '--help' '--build-as-user' '--build-command' '--from-configuration' '--generate-salt' '--no-generate-salt' '--makeflags-jobs' '--no-makeflags-jobs' '--mirror' '--multilib' '--no-multilib' '--packager' '--repository' '--sign-key' '--sign-target' '--web-port' '--web-unix-socket')
_shtab_ahriman_repo_init_option_strings=('-h' '--help' '--build-as-user' '--build-command' '--from-configuration' '--generate-salt' '--no-generate-salt' '--makeflags-jobs' '--no-makeflags-jobs' '--mirror' '--multilib' '--no-multilib' '--packager' '--repository' '--sign-key' '--sign-target' '--web-port' '--web-unix-socket')
_shtab_ahriman_repo_setup_option_strings=('-h' '--help' '--build-as-user' '--build-command' '--from-configuration' '--generate-salt' '--no-generate-salt' '--makeflags-jobs' '--no-makeflags-jobs' '--mirror' '--multilib' '--no-multilib' '--packager' '--repository' '--sign-key' '--sign-target' '--web-port' '--web-unix-socket')
_shtab_ahriman_setup_option_strings=('-h' '--help' '--build-as-user' '--build-command' '--from-configuration' '--generate-salt' '--no-generate-salt' '--makeflags-jobs' '--no-makeflags-jobs' '--mirror' '--multilib' '--no-multilib' '--packager' '--repository' '--sign-key' '--sign-target' '--web-port' '--web-unix-socket')
_shtab_ahriman_service_shell_option_strings=('-h' '--help')
_shtab_ahriman_shell_option_strings=('-h' '--help')
_shtab_ahriman_user_add_option_strings=('-h' '--help' '-p' '--password' '-r' '--role' '-s' '--secure')
_shtab_ahriman_user_add_option_strings=('-h' '--help' '--key' '--packager' '-p' '--password' '-r' '--role')
_shtab_ahriman_user_list_option_strings=('-h' '--help' '-e' '--exit-code' '-r' '--role')
_shtab_ahriman_user_remove_option_strings=('-h' '--help')
_shtab_ahriman_web_option_strings=('-h' '--help')
@ -133,6 +133,7 @@ _shtab_ahriman_search___info_nargs=0
_shtab_ahriman_search___no_info_nargs=0
_shtab_ahriman_help__h_nargs=0
_shtab_ahriman_help___help_nargs=0
_shtab_ahriman_help_commands_unsafe_pos_0_nargs=*
_shtab_ahriman_help_commands_unsafe__h_nargs=0
_shtab_ahriman_help_commands_unsafe___help_nargs=0
_shtab_ahriman_help_updates__h_nargs=0
@ -412,30 +413,40 @@ _shtab_ahriman_key_import__h_nargs=0
_shtab_ahriman_key_import___help_nargs=0
_shtab_ahriman_service_setup__h_nargs=0
_shtab_ahriman_service_setup___help_nargs=0
_shtab_ahriman_service_setup___generate_salt_nargs=0
_shtab_ahriman_service_setup___no_generate_salt_nargs=0
_shtab_ahriman_service_setup___makeflags_jobs_nargs=0
_shtab_ahriman_service_setup___no_makeflags_jobs_nargs=0
_shtab_ahriman_service_setup___multilib_nargs=0
_shtab_ahriman_service_setup___no_multilib_nargs=0
_shtab_ahriman_init__h_nargs=0
_shtab_ahriman_init___help_nargs=0
_shtab_ahriman_init___generate_salt_nargs=0
_shtab_ahriman_init___no_generate_salt_nargs=0
_shtab_ahriman_init___makeflags_jobs_nargs=0
_shtab_ahriman_init___no_makeflags_jobs_nargs=0
_shtab_ahriman_init___multilib_nargs=0
_shtab_ahriman_init___no_multilib_nargs=0
_shtab_ahriman_repo_init__h_nargs=0
_shtab_ahriman_repo_init___help_nargs=0
_shtab_ahriman_repo_init___generate_salt_nargs=0
_shtab_ahriman_repo_init___no_generate_salt_nargs=0
_shtab_ahriman_repo_init___makeflags_jobs_nargs=0
_shtab_ahriman_repo_init___no_makeflags_jobs_nargs=0
_shtab_ahriman_repo_init___multilib_nargs=0
_shtab_ahriman_repo_init___no_multilib_nargs=0
_shtab_ahriman_repo_setup__h_nargs=0
_shtab_ahriman_repo_setup___help_nargs=0
_shtab_ahriman_repo_setup___generate_salt_nargs=0
_shtab_ahriman_repo_setup___no_generate_salt_nargs=0
_shtab_ahriman_repo_setup___makeflags_jobs_nargs=0
_shtab_ahriman_repo_setup___no_makeflags_jobs_nargs=0
_shtab_ahriman_repo_setup___multilib_nargs=0
_shtab_ahriman_repo_setup___no_multilib_nargs=0
_shtab_ahriman_setup__h_nargs=0
_shtab_ahriman_setup___help_nargs=0
_shtab_ahriman_setup___generate_salt_nargs=0
_shtab_ahriman_setup___no_generate_salt_nargs=0
_shtab_ahriman_setup___makeflags_jobs_nargs=0
_shtab_ahriman_setup___no_makeflags_jobs_nargs=0
_shtab_ahriman_setup___multilib_nargs=0
@ -450,8 +461,6 @@ _shtab_ahriman_shell__v_nargs=0
_shtab_ahriman_shell___verbose_nargs=0
_shtab_ahriman_user_add__h_nargs=0
_shtab_ahriman_user_add___help_nargs=0
_shtab_ahriman_user_add__s_nargs=0
_shtab_ahriman_user_add___secure_nargs=0
_shtab_ahriman_user_list__h_nargs=0
_shtab_ahriman_user_list___help_nargs=0
_shtab_ahriman_user_list__e_nargs=0

View File

@ -58,9 +58,9 @@ _shtab_ahriman_commands() {
"service-config-validate:validate configuration and print found errors"
"service-key-import:import PGP key from public sources to the repository user"
"service-setup:create initial service configuration, requires root"
"service-shell:drop into python shell while having created application"
"service-shell:drop into python shell"
"setup:create initial service configuration, requires root"
"shell:drop into python shell while having created application"
"shell:drop into python shell"
"sign:(re-)sign packages and repository database according to current settings"
"status:request status of the package"
"status-update:update package status on the status page"
@ -95,6 +95,7 @@ _shtab_ahriman_add_options=(
{-n,--now}"[run update function after]"
"*"{-y,--refresh}"[download fresh package databases from the mirror before actions, -yy to force refresh even if up to date]"
{-s,--source}"[explicitly specify the package source for this command]:source:(auto archive aur directory local remote repository)"
{-u,--username}"[build as user]:username:"
"(*):package source (base name, path to local files, remote URL):"
)
@ -151,7 +152,7 @@ _shtab_ahriman_help_options=(
_shtab_ahriman_help_commands_unsafe_options=(
"(- : *)"{-h,--help}"[show this help message and exit]"
"--command[instead of showing commands, just test command line for unsafe subcommand and return 0 in case if command is safe and 1 otherwise]:command:"
"(*)::instead of showing commands, just test command line for unsafe subcommand and return 0 in case if command is safe and 1 otherwise:"
)
_shtab_ahriman_help_updates_options=(
@ -168,6 +169,7 @@ _shtab_ahriman_init_options=(
"--build-as-user[force makepkg user to the specific one]:build_as_user:"
"--build-command[build command prefix]:build_command:"
"--from-configuration[path to default devtools pacman configuration]:from_configuration:"
{--generate-salt,--no-generate-salt}"[generate salt for user passwords]:generate_salt:"
{--makeflags-jobs,--no-makeflags-jobs}"[append MAKEFLAGS variable with parallelism set to number of cores]:makeflags_jobs:"
"--mirror[use the specified explicitly mirror instead of including mirrorlist]:mirror:"
{--multilib,--no-multilib}"[add or do not multilib repository]:multilib:"
@ -192,6 +194,7 @@ _shtab_ahriman_package_add_options=(
{-n,--now}"[run update function after]"
"*"{-y,--refresh}"[download fresh package databases from the mirror before actions, -yy to force refresh even if up to date]"
{-s,--source}"[explicitly specify the package source for this command]:source:(auto archive aur directory local remote repository)"
{-u,--username}"[build as user]:username:"
"(*):package source (base name, path to local files, remote URL):"
)
@ -227,6 +230,7 @@ _shtab_ahriman_package_update_options=(
{-n,--now}"[run update function after]"
"*"{-y,--refresh}"[download fresh package databases from the mirror before actions, -yy to force refresh even if up to date]"
{-s,--source}"[explicitly specify the package source for this command]:source:(auto archive aur directory local remote repository)"
{-u,--username}"[build as user]:username:"
"(*):package source (base name, path to local files, remote URL):"
)
@ -263,6 +267,7 @@ _shtab_ahriman_rebuild_options=(
"--from-database[read packages from database instead of filesystem. This feature in particular is required in case if you would like to restore repository from another repository instance. Note, however, that in order to restore packages you need to have original ahriman instance run with web service and have run repo-update at least once.]"
{-e,--exit-code}"[return non-zero exit status if result is empty]"
{-s,--status}"[filter packages by status. Requires --from-database to be set]:status:(unknown pending building failed success)"
{-u,--username}"[build as user]:username:"
)
_shtab_ahriman_remove_options=(
@ -331,6 +336,7 @@ _shtab_ahriman_repo_init_options=(
"--build-as-user[force makepkg user to the specific one]:build_as_user:"
"--build-command[build command prefix]:build_command:"
"--from-configuration[path to default devtools pacman configuration]:from_configuration:"
{--generate-salt,--no-generate-salt}"[generate salt for user passwords]:generate_salt:"
{--makeflags-jobs,--no-makeflags-jobs}"[append MAKEFLAGS variable with parallelism set to number of cores]:makeflags_jobs:"
"--mirror[use the specified explicitly mirror instead of including mirrorlist]:mirror:"
{--multilib,--no-multilib}"[add or do not multilib repository]:multilib:"
@ -349,6 +355,7 @@ _shtab_ahriman_repo_rebuild_options=(
"--from-database[read packages from database instead of filesystem. This feature in particular is required in case if you would like to restore repository from another repository instance. Note, however, that in order to restore packages you need to have original ahriman instance run with web service and have run repo-update at least once.]"
{-e,--exit-code}"[return non-zero exit status if result is empty]"
{-s,--status}"[filter packages by status. Requires --from-database to be set]:status:(unknown pending building failed success)"
{-u,--username}"[build as user]:username:"
)
_shtab_ahriman_repo_remove_unknown_options=(
@ -371,6 +378,7 @@ _shtab_ahriman_repo_setup_options=(
"--build-as-user[force makepkg user to the specific one]:build_as_user:"
"--build-command[build command prefix]:build_command:"
"--from-configuration[path to default devtools pacman configuration]:from_configuration:"
{--generate-salt,--no-generate-salt}"[generate salt for user passwords]:generate_salt:"
{--makeflags-jobs,--no-makeflags-jobs}"[append MAKEFLAGS variable with parallelism set to number of cores]:makeflags_jobs:"
"--mirror[use the specified explicitly mirror instead of including mirrorlist]:mirror:"
{--multilib,--no-multilib}"[add or do not multilib repository]:multilib:"
@ -413,6 +421,7 @@ _shtab_ahriman_repo_update_options=(
{-e,--exit-code}"[return non-zero exit status if result is empty]"
{--local,--no-local}"[enable or disable checking of local packages for updates]:local:"
{--manual,--no-manual}"[include or exclude manual updates]:manual:"
{-u,--username}"[build as user]:username:"
{--vcs,--no-vcs}"[fetch actual version of VCS packages]:vcs:"
"*"{-y,--refresh}"[download fresh package databases from the mirror before actions, -yy to force refresh even if up to date]"
"(*)::filter check by package base:"
@ -460,6 +469,7 @@ _shtab_ahriman_service_setup_options=(
"--build-as-user[force makepkg user to the specific one]:build_as_user:"
"--build-command[build command prefix]:build_command:"
"--from-configuration[path to default devtools pacman configuration]:from_configuration:"
{--generate-salt,--no-generate-salt}"[generate salt for user passwords]:generate_salt:"
{--makeflags-jobs,--no-makeflags-jobs}"[append MAKEFLAGS variable with parallelism set to number of cores]:makeflags_jobs:"
"--mirror[use the specified explicitly mirror instead of including mirrorlist]:mirror:"
{--multilib,--no-multilib}"[add or do not multilib repository]:multilib:"
@ -481,6 +491,7 @@ _shtab_ahriman_setup_options=(
"--build-as-user[force makepkg user to the specific one]:build_as_user:"
"--build-command[build command prefix]:build_command:"
"--from-configuration[path to default devtools pacman configuration]:from_configuration:"
{--generate-salt,--no-generate-salt}"[generate salt for user passwords]:generate_salt:"
{--makeflags-jobs,--no-makeflags-jobs}"[append MAKEFLAGS variable with parallelism set to number of cores]:makeflags_jobs:"
"--mirror[use the specified explicitly mirror instead of including mirrorlist]:mirror:"
{--multilib,--no-multilib}"[add or do not multilib repository]:multilib:"
@ -529,6 +540,7 @@ _shtab_ahriman_update_options=(
{-e,--exit-code}"[return non-zero exit status if result is empty]"
{--local,--no-local}"[enable or disable checking of local packages for updates]:local:"
{--manual,--no-manual}"[include or exclude manual updates]:manual:"
{-u,--username}"[build as user]:username:"
{--vcs,--no-vcs}"[fetch actual version of VCS packages]:vcs:"
"*"{-y,--refresh}"[download fresh package databases from the mirror before actions, -yy to force refresh even if up to date]"
"(*)::filter check by package base:"
@ -536,9 +548,10 @@ _shtab_ahriman_update_options=(
_shtab_ahriman_user_add_options=(
"(- : *)"{-h,--help}"[show this help message and exit]"
"--key[optional PGP key used by this user. The private key must be imported]:key:"
"--packager[optional packager id used for build process in form of \`Name Surname \<mail\@example.com\>\`]:packager:"
{-p,--password}"[user password. Blank password will be treated as empty password, which is in particular must be used for OAuth2 authorization type.]:password:"
{-r,--role}"[user access level]:role:(unauthorized read reporter full)"
{-s,--secure}"[set file permissions to user-only]"
":username for web service:"
)

View File

@ -87,7 +87,6 @@ Settings for signing packages or repository. Group name can refer to architectur
* ``target`` - configuration flag to enable signing, space separated list of strings, required. Allowed values are ``package`` (sign each package separately), ``repository`` (sign repository database file).
* ``key`` - default PGP key, string, required. This key will also be used for database signing if enabled.
* ``key_*`` settings - PGP key which will be used for specific packages, string, optional. For example, if there is ``key_yay`` option the specified key will be used for yay package and default key for others.
``web:*`` groups
----------------

View File

@ -196,7 +196,7 @@ Alternatively you can create full-diff patches, which are calculated by using ``
.. code-block:: shell
sudo -u ahriman ahriman patch-set-add /path/to/local/directory/with/PKGBUILD
sudo -u ahriman ahriman patch-set-add /path/to/local/directory/with/PKGBUILD
The last command will calculate diff from current tree to the ``HEAD`` and will store it locally. Patches will be applied on any package actions (e.g. it can be used for dependency management).
@ -862,12 +862,13 @@ How to enable basic authorization
yay -S --asdeps python-aiohttp-security python-aiohttp-session python-cryptography
#.
Configure the service to enable authorization:
Configure the service to enable authorization (``salt`` can be generated as any random string):
.. code-block:: ini
[auth]
target = configuration
salt = somerandomstring
#.
In order to provide access for reporting from application instances you can (recommended way) use unix sockets by configuring the following (note, that it requires ``python-requests-unixsocket`` package to be installed):
@ -933,7 +934,7 @@ How to enable OAuth authorization
Configure ``oauth_provider`` and ``oauth_scopes`` in case if you would like to use different from Google provider. Scope must grant access to user email. ``web.address`` is required to make callback URL available from internet.
#.
Create service user:
If you are not going to use unix socket, you also need to create service user (remember to set ``auth.salt`` option before):
.. code-block:: shell

View File

@ -12,7 +12,7 @@ Features
* VCS packages support.
* Official repository support.
* Ability to patch AUR packages and even create package from local PKGBUILDs.
* Sign support with gpg (repository, package, per package settings).
* Sign support with gpg (repository, package), multiple packagers support.
* Triggers for repository updates, e.g. synchronization to remote services (rsync, s3 and github) and report generation (email, html, telegram).
* Repository status interface with optional authorization and control options.

View File

@ -64,7 +64,7 @@ Initial setup
.. code-block:: shell
echo 'Cmnd_Alias CARCHBUILD_CMD = /usr/local/bin/ahriman-x86_64-build *' | tee -a /etc/sudoers.d/ahriman
echo 'ahriman ALL=(ALL) NOPASSWD: CARCHBUILD_CMD' | tee -a /etc/sudoers.d/ahriman
echo 'ahriman ALL=(ALL) NOPASSWD:SETENV: CARCHBUILD_CMD' | tee -a /etc/sudoers.d/ahriman
chmod 400 /etc/sudoers.d/ahriman
This command supports several arguments, kindly refer to its help message.

BIN
github-logo.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 62 KiB

View File

@ -1,7 +1,7 @@
# Maintainer: Evgeniy Alekseev
pkgname='ahriman'
pkgver=2.9.0
pkgver=2.10.1
pkgrel=1
pkgdesc="ArcH linux ReposItory MANager"
arch=('any')

View File

@ -63,6 +63,8 @@
<table id="packages" class="table table-striped table-hover"
data-export-options='{"fileName": "packages"}'
data-filter-control="true"
data-filter-control-visible="false"
data-page-list="[10, 25, 50, 100, all]"
data-page-size="10"
data-pagination="true"
@ -72,6 +74,7 @@
data-show-columns-search="true"
data-show-columns-toggle-all="true"
data-show-export="true"
data-show-filter-control-switch="true"
data-show-fullscreen="true"
data-show-search-clear-button="true"
data-sortable="true"
@ -82,13 +85,14 @@
<thead class="table-primary">
<tr>
<th data-checkbox="true"></th>
<th data-sortable="true" data-switchable="false" data-field="base">package base</th>
<th data-sortable="true" data-field="version">version</th>
<th data-sortable="true" data-field="packages">packages</th>
<th data-sortable="true" data-visible="false" data-field="groups">groups</th>
<th data-sortable="true" data-visible="false" data-field="licenses">licenses</th>
<th data-sortable="true" data-field="timestamp">last update</th>
<th data-sortable="true" data-cell-style="statusFormat" data-field="status">status</th>
<th data-sortable="true" data-switchable="false" data-field="base" data-filter-control="input" data-filter-control-placeholder="(any base)">package base</th>
<th data-sortable="true" data-field="version" data-filter-control="input" data-filter-control-placeholder="(any version)">version</th>
<th data-sortable="true" data-field="packages" data-filter-control="input" data-filter-control-placeholder="(any package)">packages</th>
<th data-sortable="true" data-visible="false" data-field="groups" data-filter-control="select" data-filter-data="func:filterListGroups" data-filter-custom-search="filterList" data-filter-control-placeholder="(any group)">groups</th>
<th data-sortable="true" data-visible="false" data-field="licenses" data-filter-control="select" data-filter-data="func:filterListLicenses" data-filter-custom-search="filterList" data-filter-control-placeholder="(any license)">licenses</th>
<th data-sortable="true" data-visible="false" data-field="packager" data-filter-control="select" data-filter-custom-search="filterContains" data-filter-control-placeholder="(any packager)">packager</th>
<th data-sortable="true" data-field="timestamp" data-filter-control="input" data-filter-custom-search="filterDateRange" data-filter-control-placeholder="(any date)">last update</th>
<th data-sortable="true" data-cell-style="statusFormat" data-field="status" data-filter-control="select" data-filter-control-placeholder="(any status)">status</th>
</tr>
</thead>
</table>

View File

@ -84,7 +84,7 @@
showSuccess("Success", `Key ${key} has been imported`);
},
error: (jqXHR, _, errorThrown) => {
const message = _ => { return `Could not import key ${key} from ${server}`; };
const message = _ => `Could not import key ${key} from ${server}`;
showFailure("Action failed", message, jqXHR, errorThrown);
},
});

View File

@ -60,8 +60,8 @@
const packages = packageInput.val();
if (packages) {
packageAddModal.modal("hide");
const onSuccess = update => { return `Packages ${update} have been added`; };
const onFailure = error => { return `Package addition failed: ${error}`; };
const onSuccess = update => `Packages ${update} have been added`;
const onFailure = error => `Package addition failed: ${error}`;
doPackageAction("/api/v1/service/add", [packages], onSuccess, onFailure);
}
}
@ -70,8 +70,8 @@
const packages = packageInput.val();
if (packages) {
packageAddModal.modal("hide");
const onSuccess = update => { return `Packages ${update} have been requested`; };
const onFailure = error => { return `Package request failed: ${error}`; };
const onSuccess = update => `Packages ${update} have been requested`;
const onFailure = error => `Package request failed: ${error}`;
doPackageAction("/api/v1/service/request", [packages], onSuccess, onFailure);
}
}

View File

@ -61,7 +61,7 @@
error: (jqXHR, _, errorThrown) => {
// show failed modal in case if first time loading
if (isPackageBaseSet) {
const message = error => { return `Could not load package ${packageBase} logs: ${error}`; };
const message = error => `Could not load package ${packageBase} logs: ${error}`;
showFailure("Load failure", message, jqXHR, errorThrown);
}
},

View File

@ -33,8 +33,8 @@
const packages = dependencyInput.val();
if (packages) {
packageRebuildModal.modal("hide");
const onSuccess = update => { return `Repository rebuild has been run for packages which depend on ${update}`; };
const onFailure = error => { return `Repository rebuild failed: ${error}`; };
const onSuccess = update => `Repository rebuild has been run for packages which depend on ${update}`;
const onFailure = error => `Repository rebuild failed: ${error}`;
doPackageAction("/api/v1/service/rebuild", [packages], onSuccess, onFailure);
}
}

View File

@ -15,6 +15,25 @@
table.bootstrapTable(method, {field: "id", values: [data.id]});
} else showLogs(data.id);
});
table.on("created-controls.bs.table", () => {
const pickerInput = $(".bootstrap-table-filter-control-timestamp");
pickerInput.daterangepicker({
autoUpdateInput: false,
locale: {
cancelLabel: "Clear",
},
});
pickerInput.on("apply.daterangepicker", (event, picker) => {
pickerInput.val(`${picker.startDate.format("YYYY-MM-DD")} - ${picker.endDate.format("YYYY-MM-DD")}`);
table.bootstrapTable("triggerSearch");
});
pickerInput.on("cancel.daterangepicker", () => {
pickerInput.val("");
table.bootstrapTable("triggerSearch");
});
});
const repositoryBadge = $("#badge-repository");
const statusBadge = $("#badge-status");
@ -37,21 +56,21 @@
}
function getSelection() {
return table.bootstrapTable("getSelections").map(row => { return row.id; });
return table.bootstrapTable("getSelections").map(row => row.id);
}
function removePackages() {
const onSuccess = update => { return `Packages ${update} have been removed`; };
const onFailure = error => { return `Could not remove packages: ${error}`; };
const onSuccess = update => `Packages ${update} have been removed`;
const onFailure = error => `Could not remove packages: ${error}`;
doPackageAction("/api/v1/service/remove", getSelection(), onSuccess, onFailure);
}
function updatePackages() {
const currentSelection = getSelection();
const [url, onSuccess] = currentSelection.length === 0
? ["/api/v1/service/update", _ => { return "Repository update has been run"; }]
: ["/api/v1/service/add", update => { return `Run update for packages ${update}`; }];
const onFailure = error => { return `Packages update failed: ${error}`; };
? ["/api/v1/service/update", _ => "Repository update has been run"]
: ["/api/v1/service/add", update => `Run update for packages ${update}`];
const onFailure = error => `Packages update failed: ${error}`;
doPackageAction(url, currentSelection, onSuccess, onFailure);
}
@ -81,13 +100,13 @@
success: response => {
const extractListProperties = (description, property) => {
return Object.values(description.packages)
.map(pkg => { return pkg[property]; })
.reduce((left, right) => { return left.concat(right); }, []);
.map(pkg => pkg[property])
.reduce((left, right) => left.concat(right), []);
};
const listToTable = data => {
return Array.from(new Set(data))
.sort()
.map(entry => { return safe(entry); })
.map(entry => safe(entry))
.join("<br>");
};
@ -98,6 +117,7 @@
id: package_base,
base: web_url ? `<a href="${safe(web_url)}" title="${safe(package_base)}">${safe(package_base)}</a>` : safe(package_base),
version: safe(description.package.version),
packager: description.package.packager ? safe(description.package.packager) : "",
packages: listToTable(Object.keys(description.package.packages)),
groups: listToTable(extractListProperties(description.package, "groups")),
licenses: listToTable(extractListProperties(description.package, "licenses")),
@ -120,8 +140,8 @@
table.bootstrapTable("hideLoading");
} else {
// other errors
const messaga = error => { return `Could not load list of packages: ${error}`; };
showFailure("Load failure", messaga, jqXHR, errorThrown);
const message = error => `Could not load list of packages: ${error}`;
showFailure("Load failure", message, jqXHR, errorThrown);
}
hideControls(true);
},
@ -157,6 +177,18 @@
return {classes: cellClass(value)};
}
function filterListGroups() {
return extractDataList(table.bootstrapTable("getData"), "groups");
}
function filterListLicenses() {
return extractDataList(table.bootstrapTable("getData"), "licenses");
}
function filterListPackagers() {
return extractDataList(table.bootstrapTable("getData"), "packager");
}
$(() => {
table.bootstrapTable({});
statusBadge.popover();

View File

@ -31,6 +31,8 @@ SigLevel = Database{% if has_repo_signed %}Required{% else %}Never{% endif %} Pa
<div class="container">
<table id="packages" class="table table-striped table-hover"
data-export-options='{"fileName": "packages"}'
data-filter-control="true"
data-filter-control-visible="false"
data-page-list="[10, 25, 50, 100, all]"
data-page-size="10"
data-pagination="true"
@ -40,6 +42,7 @@ SigLevel = Database{% if has_repo_signed %}Required{% else %}Never{% endif %} Pa
data-show-columns-search="true"
data-show-columns-toggle-all="true"
data-show-export="true"
data-show-filter-control-switch="true"
data-show-fullscreen="true"
data-show-search-clear-button="true"
data-sortable="true"
@ -48,17 +51,17 @@ SigLevel = Database{% if has_repo_signed %}Required{% else %}Never{% endif %} Pa
data-toggle="table">
<thead class="table-primary">
<tr>
<th data-sortable="true" data-switchable="false">package</th>
<th data-sortable="true">version</th>
<th data-sortable="true" data-visible="false">architecture</th>
<th data-sortable="true" data-visible="false">description</th>
<th data-sortable="true" data-visible="false">upstream url</th>
<th data-sortable="true" data-visible="false">licenses</th>
<th data-sortable="true" data-visible="false">groups</th>
<th data-sortable="true" data-visible="false">depends</th>
<th data-sortable="true">archive size</th>
<th data-sortable="true">installed size</th>
<th data-sortable="true">build date</th>
<th data-sortable="true" data-switchable="false" data-field="name" data-filter-control="input" data-filter-control-placeholder="(any package)">package</th>
<th data-sortable="true" data-field="version" data-filter-control="input" data-filter-control-placeholder="(any version)">version</th>
<th data-sortable="true" data-visible="false" data-field="architecture" data-filter-control="select" data-filter-control-placeholder="(any arch)">architecture</th>
<th data-sortable="true" data-visible="false" data-field="description" data-filter-control="input" data-filter-control-placeholder="(any description)">description</th>
<th data-sortable="true" data-visible="false" data-field="url">upstream url</th>
<th data-sortable="true" data-visible="false" data-field="licenses" data-filter-control="select" data-filter-data="func:filterListLicenses" data-filter-custom-search="filterList" data-filter-control-placeholder="(any license)">licenses</th>
<th data-sortable="true" data-visible="false" data-field="groups" data-filter-control="select" data-filter-data="func:filterListGroups" data-filter-custom-search="filterList" data-filter-control-placeholder="(any group)">groups</th>
<th data-sortable="true" data-visible="false" data-field="depends" data-filter-control="select" data-filter-data="func:filterListDepends" data-filter-custom-search="filterList" data-filter-control-placeholder="(any depends)">depends</th>
<th data-sortable="true" data-field="archive_size">archive size</th>
<th data-sortable="true" data-field="installed_size">installed size</th>
<th data-sortable="true" data-field="timestamp" data-filter-control="input" data-filter-custom-search="filterDateRange" data-filter-control-placeholder="(any date)">build date</th>
</tr>
</thead>
@ -96,6 +99,27 @@ SigLevel = Database{% if has_repo_signed %}Required{% else %}Never{% endif %} Pa
</div>
<script>
const table = $("#packages");
table.on("created-controls.bs.table", () => {
const pickerInput = $(".bootstrap-table-filter-control-timestamp");
pickerInput.daterangepicker({
autoUpdateInput: false,
locale: {
cancelLabel: "Clear",
},
});
pickerInput.on("apply.daterangepicker", (event, picker) => {
pickerInput.val(`${picker.startDate.format("YYYY-MM-DD")} - ${picker.endDate.format("YYYY-MM-DD")}`);
table.bootstrapTable("triggerSearch");
});
pickerInput.on("cancel.daterangepicker", () => {
pickerInput.val("");
table.bootstrapTable("triggerSearch");
});
});
const pacmanConf = $("#pacman-conf");
const pacmanConfCopyButton = $("#copy-btn");
@ -103,6 +127,18 @@ SigLevel = Database{% if has_repo_signed %}Required{% else %}Never{% endif %} Pa
const conf = pacmanConf.text();
await copyToClipboard(conf, pacmanConfCopyButton);
}
function filterListDepends() {
return extractDataList(table.bootstrapTable("getData"), "depends");
}
function filterListGroups() {
return extractDataList(table.bootstrapTable("getData"), "groups");
}
function filterListLicenses() {
return extractDataList(table.bootstrapTable("getData"), "licenses");
}
</script>
</body>

View File

@ -1,16 +1,21 @@
<script src="https://cdn.jsdelivr.net/npm/jquery/dist/jquery.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/jquery@3.7.0/dist/jquery.min.js" integrity="sha384-NXgwF8Kv9SSAr+jemKKcbvQsz+teULH/a5UNJvZc6kP47hZgl62M1vGnw6gHQhb1" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://unpkg.com/tableexport.jquery.plugin/tableExport.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/moment@2.29.4/moment.min.js" integrity="sha384-8hHkOkbWN1TLWwet/jpbJ0zbx3FJDeYJgQ8dX1mRrv/vfCfHCqFSFZYCgaMML3z9" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/daterangepicker@3.1.0/daterangepicker.min.js" integrity="sha384-u4eJN1VWrTf/FnYYQJo2kqJyVxEQf5UmWY4iUcNAoLenOEtEuCkfwc5bKvZOWBi5" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://unpkg.com/jquery-resizable-columns@0.2.3/dist/jquery.resizableColumns.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/tableexport.jquery.plugin@1.28.0/tableExport.min.js" integrity="sha384-1Rz4Kz/y1rSWw+ZsjTcxB684XgofbO8iizY+UFIzCwFeQ+QUyhBNWBMh/STOyomI" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/@popperjs/core@2.11.6/dist/umd/popper.min.js" integrity="sha384-oBqDVmMz9ATKxIep9tiCxS/Z9fNfEXiDAYTujMAeBAsjFuCZSmKbSSUnQlmh/jp3" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.2.2/dist/js/bootstrap.min.js" integrity="sha384-IDwe1+LCz02ROU9k972gdyvl+AESN10+x7tBKgc9I5HFtuNz0wWnPclzo6p9vxnk" crossorigin="anonymous"></script>
<script src="https://unpkg.com/bootstrap-table@1.21.1/dist/bootstrap-table.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/jquery-resizable-columns@0.2.3/dist/jquery.resizableColumns.min.js" integrity="sha384-IazMVNyYoUNx6357fWJoqtHYUWWCNHIXxFVtbpVgvImQNWuRP2WbHPaIb3QF8j97" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://unpkg.com/bootstrap-table@1.21.1/dist/extensions/export/bootstrap-table-export.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@popperjs/core@2.11.8/dist/umd/popper.min.js" integrity="sha384-I7E8VVD/ismYTF4hNIPjVp/Zjvgyol6VFvRkX/vR+Vc4jQkC+hVqc2pM8ODewa9r" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.2.3/dist/js/bootstrap.min.js" integrity="sha384-cuYeSxntonz0PPNlHhBs68uyIAVpIIOZZ5JqeqvYYIcEL727kskC66kF92t6Xl2V" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap-table@1.22.1/dist/bootstrap-table.min.js" integrity="sha384-GVLHfbEvuGA/RFiQ3MK2ClEJkWYJXABg55t9LpoDPZFGIsSq8xhFlQydm5poV2jW" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://unpkg.com/bootstrap-table@1.21.1/dist/extensions/resizable/bootstrap-table-resizable.js"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap-table@1.21.4/dist/extensions/export/bootstrap-table-export.min.js" integrity="sha384-jeldDadm+qM2RwGER3qVqxFgWVpAEJ7Jie+0rlYj8ni3KkQA654T8TSXDtol022X" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap-table@1.21.4/dist/extensions/resizable/bootstrap-table-resizable.js" integrity="sha384-wd8Vc6Febikdnsnk9vthRWRvMwffw246vhqiqNO3aSNe1maTEA07Vh3zAQiSyDji" crossorigin="anonymous" type="application/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap-table@1.21.4/dist/extensions/filter-control/bootstrap-table-filter-control.js" integrity="sha384-B6xNXlSOaOFxjlKo9OW3htbox+9/DcaEcjPPEi1+pTMwH5Tzc/s2wNTYriHz7Tb8" crossorigin="anonymous" type="application/javascript"></script>
<script>
async function copyToClipboard(text, button) {
@ -38,4 +43,28 @@
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;");
}
function extractDataList(data, column) {
const elements = data.flatMap(row => row[column].split("<br>")).filter(v => v); // remove empty elements from array
return Array.from(new Set(elements)).sort();
}
function filterContains(text, value) {
return value.includes(text.toLowerCase().trim());
}
function filterDateRange(text, value) {
const asOfStartOfDay = date => date.setUTCHours(0, 0, 0, 0);
const [minDate, maxDate] = text.split(" - ");
const buildDate = asOfStartOfDay(new Date(value));
return (buildDate >= asOfStartOfDay(new Date(minDate))) && (buildDate <= asOfStartOfDay(new Date(maxDate)));
}
function filterList(index, value, field, data) {
const dataList = extractDataList(data, field);
// the library removes all symbols from string, so it is just string
return value.includes(dataList[index].toLowerCase());
}
</script>

View File

@ -1,11 +1,15 @@
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap@5.2.2/dist/css/bootstrap.min.css" integrity="sha384-Zenh87qX5JnK2Jl0vWa8Ck2rdkQ2Bzep5IDxbcnCeuOxjzrPF/et3URy9Bv1WTRi" crossorigin="anonymous" type="text/css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.10.2/font/bootstrap-icons.css" type="text/css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap@5.2.3/dist/css/bootstrap.min.css" integrity="sha384-rbsA2VBKQhggwzxH7pPCaAqO46MgnOM80zW1RWuH61DGLwZJEdK2Kadq2F9CUG65" crossorigin="anonymous" type="text/css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.10.4/font/bootstrap-icons.css" integrity="sha384-LrVLJJYk9OiJmjNDakUBU7kS9qCT8wk1j2OU7ncpsfB3QS37UPdkCuq3ZD1MugNY" crossorigin="anonymous" type="text/css">
<link rel="stylesheet" href="https://unpkg.com/bootstrap-table@1.21.1/dist/bootstrap-table.min.css" type="text/css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap-table@1.21.4/dist/bootstrap-table.min.css" integrity="sha384-pTEAhytv7JmEG2D7qiW5gY0lI5EKZ9n3CNmj6Qp+U3qhnmH2qnnN9KJbVwbtMHN0" crossorigin="anonymous" type="text/css">
<link rel="stylesheet" href="https://unpkg.com/jquery-resizable-columns@0.2.3/dist/jquery.resizableColumns.css" type="text/css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/jquery-resizable-columns@0.2.3/dist/jquery.resizableColumns.css" integrity="sha384-1sLxvR8mXzjhvFY9f8mzXl97DNLepeZ0PnRiMMdm/rQsKjsrPZPJxYle2wwT2PMg" crossorigin="anonymous" type="text/css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootswatch@5.2.2/dist/cosmo/bootstrap.min.css" integrity="sha256-5t++JZpgVLzo9vF7snO5Qw0y3fA5/NkoJENWB7kpg0E=" crossorigin="anonymous" type="text/css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap-table@1.21.4/dist/extensions/filter-control/bootstrap-table-filter-control.css" integrity="sha384-4Glx18jZ0Un+yDG6KUpYJ/af8hkssJ02jRASuFv23gfCl0mTXaVMPI9cB4cn3GvE" crossorigin="anonymous" type="text/css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootswatch@5.2.3/dist/cosmo/bootstrap.min.css" integrity="sha384-P1PBFVifKf1Ww0gS5B8A0siIeDpcFd4uU7S68LA1XMdE0R+y1WN3DR4HcLc9csRC" crossorigin="anonymous" type="text/css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/daterangepicker@3.1.0/daterangepicker.css" integrity="sha384-zLkQsiLfAQqGeIJeKLC+rcCR1YoYaQFLCL7cLDUoKE1ajKJzySpjzWGfYS2vjSG+" crossorigin="anonymous" type="text/css">
<style>
.pre-scrollable {

View File

@ -27,7 +27,7 @@ from typing import TypeVar
from ahriman import version
from ahriman.application import handlers
from ahriman.core.util import enum_values
from ahriman.core.util import enum_values, extract_user
from ahriman.models.action import Action
from ahriman.models.build_status import BuildStatusEnum
from ahriman.models.log_handler import LogHandler
@ -187,8 +187,8 @@ def _set_help_commands_unsafe_parser(root: SubParserAction) -> argparse.Argument
"""
parser = root.add_parser("help-commands-unsafe", help="list unsafe commands",
description="list unsafe commands as defined in default args", formatter_class=_formatter)
parser.add_argument("--command", help="instead of showing commands, just test command line for unsafe subcommand "
"and return 0 in case if command is safe and 1 otherwise")
parser.add_argument("command", help="instead of showing commands, just test command line for unsafe subcommand "
"and return 0 in case if command is safe and 1 otherwise", nargs="*")
parser.set_defaults(handler=handlers.UnsafeCommands, architecture=[""], lock=None, report=False, quiet=True,
unsafe=True, parser=_parser)
return parser
@ -262,6 +262,7 @@ def _set_package_add_parser(root: SubParserAction) -> argparse.ArgumentParser:
action="count", default=False)
parser.add_argument("-s", "--source", help="explicitly specify the package source for this command",
type=PackageSource, choices=enum_values(PackageSource), default=PackageSource.Auto)
parser.add_argument("-u", "--username", help="build as user", default=extract_user())
parser.set_defaults(handler=handlers.Add)
return parser
@ -481,7 +482,8 @@ def _set_repo_check_parser(root: SubParserAction) -> argparse.ArgumentParser:
parser.add_argument("-y", "--refresh", help="download fresh package databases from the mirror before actions, "
"-yy to force refresh even if up to date",
action="count", default=False)
parser.set_defaults(handler=handlers.Update, dependencies=False, dry_run=True, aur=True, local=True, manual=False)
parser.set_defaults(handler=handlers.Update, dependencies=False, dry_run=True, aur=True, local=True, manual=False,
username=None)
return parser
@ -578,6 +580,7 @@ def _set_repo_rebuild_parser(root: SubParserAction) -> argparse.ArgumentParser:
parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty", action="store_true")
parser.add_argument("-s", "--status", help="filter packages by status. Requires --from-database to be set",
type=BuildStatusEnum, choices=enum_values(BuildStatusEnum))
parser.add_argument("-u", "--username", help="build as user", default=extract_user())
parser.set_defaults(handler=handlers.Rebuild)
return parser
@ -752,6 +755,7 @@ def _set_repo_update_parser(root: SubParserAction) -> argparse.ArgumentParser:
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--manual", help="include or exclude manual updates",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("-u", "--username", help="build as user", default=extract_user())
parser.add_argument("--vcs", help="fetch actual version of VCS packages",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("-y", "--refresh", help="download fresh package databases from the mirror before actions, "
@ -871,6 +875,8 @@ def _set_service_setup_parser(root: SubParserAction) -> argparse.ArgumentParser:
parser.add_argument("--build-command", help="build command prefix", default="ahriman")
parser.add_argument("--from-configuration", help="path to default devtools pacman configuration",
type=Path, default=Path("/usr") / "share" / "devtools" / "pacman.conf.d" / "extra.conf")
parser.add_argument("--generate-salt", help="generate salt for user passwords",
action=argparse.BooleanOptionalAction, default=False)
parser.add_argument("--makeflags-jobs", help="append MAKEFLAGS variable with parallelism set to number of cores",
action=argparse.BooleanOptionalAction, default=True)
parser.add_argument("--mirror", help="use the specified explicitly mirror instead of including mirrorlist")
@ -898,8 +904,7 @@ def _set_service_shell_parser(root: SubParserAction) -> argparse.ArgumentParser:
argparse.ArgumentParser: created argument parser
"""
parser = root.add_parser("service-shell", aliases=["shell"], help="invoke python shell",
description="drop into python shell while having created application",
formatter_class=_formatter)
description="drop into python shell", formatter_class=_formatter)
parser.add_argument("code", help="instead of dropping into shell, just execute the specified code", nargs="?")
parser.add_argument("-v", "--verbose", help=argparse.SUPPRESS, action="store_true")
parser.set_defaults(handler=handlers.Shell, lock=None, report=False)
@ -919,15 +924,15 @@ def _set_user_add_parser(root: SubParserAction) -> argparse.ArgumentParser:
parser = root.add_parser("user-add", help="create or update user",
description="update user for web services with the given password and role. "
"In case if password was not entered it will be asked interactively",
epilog="In case of first run (i.e. if password salt is not set yet) this action requires "
"root privileges because it performs write to filesystem configuration.",
formatter_class=_formatter)
parser.add_argument("username", help="username for web service")
parser.add_argument("--key", help="optional PGP key used by this user. The private key must be imported")
parser.add_argument("--packager", help="optional packager id used for build process in form of "
"`Name Surname <mail@example.com>`")
parser.add_argument("-p", "--password", help="user password. Blank password will be treated as empty password, "
"which is in particular must be used for OAuth2 authorization type.")
parser.add_argument("-r", "--role", help="user access level",
type=UserAccess, choices=enum_values(UserAccess), default=UserAccess.Read)
parser.add_argument("-s", "--secure", help="set file permissions to user-only", action="store_true")
parser.set_defaults(handler=handlers.Users, action=Action.Update, architecture=[""], lock=None, report=False,
quiet=True)
return parser
@ -949,8 +954,8 @@ def _set_user_list_parser(root: SubParserAction) -> argparse.ArgumentParser:
parser.add_argument("username", help="filter users by username", nargs="?")
parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty", action="store_true")
parser.add_argument("-r", "--role", help="filter users by role", type=UserAccess, choices=enum_values(UserAccess))
parser.set_defaults(handler=handlers.Users, action=Action.List, architecture=[""], lock=None, report=False, # nosec
password="", quiet=True, unsafe=True)
parser.set_defaults(handler=handlers.Users, action=Action.List, architecture=[""], lock=None, report=False,
quiet=True, unsafe=True)
return parser
@ -968,8 +973,8 @@ def _set_user_remove_parser(root: SubParserAction) -> argparse.ArgumentParser:
description="remove user from the user mapping and update the configuration",
formatter_class=_formatter)
parser.add_argument("username", help="username for web service")
parser.set_defaults(handler=handlers.Users, action=Action.Remove, architecture=[""], lock=None, report=False, # nosec
password="", quiet=True)
parser.set_defaults(handler=handlers.Users, action=Action.Remove, architecture=[""], lock=None, report=False,
quiet=True)
return parser

View File

@ -17,10 +17,12 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from collections.abc import Iterable
from collections.abc import Callable, Iterable
from ahriman.application.application.application_packages import ApplicationPackages
from ahriman.application.application.application_repository import ApplicationRepository
from ahriman.core.formatters import UpdatePrinter
from ahriman.core.tree import Tree
from ahriman.models.package import Package
from ahriman.models.result import Result
@ -37,12 +39,12 @@ class Application(ApplicationPackages, ApplicationRepository):
>>> from ahriman.models.package_source import PackageSource
>>>
>>> configuration = Configuration()
>>> application = Application("x86_64", configuration, report=True, unsafe=False)
>>> application = Application("x86_64", configuration, report=True)
>>> # add packages to build queue
>>> application.add(["ahriman"], PackageSource.AUR, without_dependencies=False)
>>> application.add(["ahriman"], PackageSource.AUR)
>>>
>>> # check for updates
>>> updates = application.updates([], aur=True, local=True, manual=True, vcs=True, log_fn=print)
>>> updates = application.updates([], aur=True, local=True, manual=True, vcs=True)
>>> # updates for specified packages
>>> application.update(updates)
@ -89,6 +91,22 @@ class Application(ApplicationPackages, ApplicationRepository):
"""
self.repository.triggers.on_stop()
def print_updates(self, packages: list[Package], *, log_fn: Callable[[str], None]) -> None:
"""
print list of packages to be built. This method will build dependency tree and print updates accordingly
Args:
packages(list[Package]): package list to be printed
log_fn(Callable[[str], None]): logger function to log updates
"""
local_versions = {package.base: package.version for package in self.repository.packages()}
tree = Tree.resolve(packages)
for level in tree:
for package in level:
UpdatePrinter(package, local_versions.get(package.base)).print(
verbose=True, log_fn=log_fn, separator=" -> ")
def with_dependencies(self, packages: list[Package], *, process_dependencies: bool) -> list[Package]:
"""
add missing dependencies to list of packages
@ -96,21 +114,25 @@ class Application(ApplicationPackages, ApplicationRepository):
Args:
packages(list[Package]): list of source packages of which dependencies have to be processed
process_dependencies(bool): if no set, dependencies will not be processed
Returns:
list[Package]: updated packages list. Packager for dependencies will be copied from
original package
"""
def missing_dependencies(source: Iterable[Package]) -> set[str]:
# build initial list of dependencies
result = set()
for package in source:
result.update(package.depends_build)
def missing_dependencies(source: Iterable[Package]) -> dict[str, str | None]:
# append list of known packages with packages which are in current sources
satisfied_packages = known_packages | {
single
for package in source
for single in package.packages_full
}
# remove ones which are already well-known
result = result.difference(known_packages)
# remove ones which are in this list already
for package in source:
result = result.difference(package.packages_full)
return result
return {
dependency: package.packager
for package in source
for dependency in package.depends_build
if dependency not in satisfied_packages
}
if not process_dependencies or not packages:
return packages
@ -119,8 +141,11 @@ class Application(ApplicationPackages, ApplicationRepository):
with_dependencies = {package.base: package for package in packages}
while missing := missing_dependencies(with_dependencies.values()):
for package_name in missing:
package = Package.from_aur(package_name, self.repository.pacman)
for package_name, username in missing.items():
package = Package.from_aur(package_name, self.repository.pacman, username)
with_dependencies[package.base] = package
# register package in local database
self.database.remote_update(package)
self.repository.reporter.set_unknown(package)
return list(with_dependencies.values())

View File

@ -55,15 +55,15 @@ class ApplicationPackages(ApplicationProperties):
dst = self.repository.paths.packages / local_path.name
shutil.copy(local_path, dst)
def _add_aur(self, source: str) -> None:
def _add_aur(self, source: str, username: str | None) -> None:
"""
add package from AUR
Args:
source(str): package base name
username(str | None): optional override of username for build process
"""
package = Package.from_aur(source, self.repository.pacman)
package = Package.from_aur(source, self.repository.pacman, username)
self.database.build_queue_insert(package)
self.database.remote_update(package)
@ -81,23 +81,24 @@ class ApplicationPackages(ApplicationProperties):
for full_path in filter(package_like, local_dir.iterdir()):
self._add_archive(str(full_path))
def _add_local(self, source: str) -> None:
def _add_local(self, source: str, username: str | None) -> None:
"""
add package from local PKGBUILDs
Args:
source(str): path to directory with local source files
username(str | None): optional override of username for build process
Raises:
UnknownPackageError: if specified package is unknown or doesn't exist
"""
if (source_dir := Path(source)).is_dir():
package = Package.from_build(source_dir, self.architecture)
package = Package.from_build(source_dir, self.architecture, username)
cache_dir = self.repository.paths.cache_for(package.base)
shutil.copytree(source_dir, cache_dir) # copy package to store in caches
Sources.init(cache_dir) # we need to run init command in directory where we do have permissions
elif (source_dir := self.repository.paths.cache_for(source)).is_dir():
package = Package.from_build(source_dir, self.architecture)
package = Package.from_build(source_dir, self.architecture, username)
else:
raise UnknownPackageError(source)
@ -122,29 +123,31 @@ class ApplicationPackages(ApplicationProperties):
for chunk in response.iter_content(chunk_size=1024):
local_file.write(chunk)
def _add_repository(self, source: str, *_: Any) -> None:
def _add_repository(self, source: str, username: str | None) -> None:
"""
add package from official repository
Args:
source(str): package base name
username(str | None): optional override of username for build process
"""
package = Package.from_official(source, self.repository.pacman)
package = Package.from_official(source, self.repository.pacman, username)
self.database.build_queue_insert(package)
self.database.remote_update(package)
def add(self, names: Iterable[str], source: PackageSource) -> None:
def add(self, names: Iterable[str], source: PackageSource, username: str | None = None) -> None:
"""
add packages for the next build
Args:
names(Iterable[str]): list of package bases to add
source(PackageSource): package source to add
username(str | None, optional): optional override of username for build process (Default value = None)
"""
for name in names:
resolved_source = source.resolve(name)
fn = getattr(self, f"_add_{resolved_source.value}")
fn(name)
fn(name, username)
def on_result(self, result: Result) -> None:
"""

View File

@ -35,7 +35,7 @@ class ApplicationProperties(LazyLogging):
repository(Repository): repository instance
"""
def __init__(self, architecture: str, configuration: Configuration, *, report: bool, unsafe: bool,
def __init__(self, architecture: str, configuration: Configuration, *, report: bool,
refresh_pacman_database: PacmanSynchronization = PacmanSynchronization.Disabled) -> None:
"""
default constructor
@ -44,12 +44,11 @@ class ApplicationProperties(LazyLogging):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
refresh_pacman_database(PacmanSynchronization, optional): pacman database synchronization level
(Default value = PacmanSynchronization.Disabled)
"""
self.configuration = configuration
self.architecture = architecture
self.database = SQLite.load(configuration)
self.repository = Repository.load(architecture, configuration, self.database, report=report, unsafe=unsafe,
self.repository = Repository.load(architecture, configuration, self.database, report=report,
refresh_pacman_database=refresh_pacman_database)

View File

@ -17,14 +17,14 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from collections.abc import Callable, Iterable
from collections.abc import Iterable
from pathlib import Path
from ahriman.application.application.application_properties import ApplicationProperties
from ahriman.core.build_tools.sources import Sources
from ahriman.core.formatters import UpdatePrinter
from ahriman.core.tree import Tree
from ahriman.models.package import Package
from ahriman.models.packagers import Packagers
from ahriman.models.result import Result
@ -83,7 +83,7 @@ class ApplicationRepository(ApplicationProperties):
if archive.filepath is None:
self.logger.warning("filepath is empty for %s", package.base)
continue # avoid mypy warning
self.repository.sign.process_sign_package(archive.filepath, package.base)
self.repository.sign.process_sign_package(archive.filepath, None)
# sign repository database if set
self.repository.sign.process_sign_repository(self.repository.repo.repo_path)
# process triggers
@ -104,14 +104,14 @@ class ApplicationRepository(ApplicationProperties):
packages: list[str] = []
for single in probe.packages:
try:
_ = Package.from_aur(single, self.repository.pacman)
_ = Package.from_aur(single, self.repository.pacman, None)
except Exception:
packages.append(single)
return packages
def unknown_local(probe: Package) -> list[str]:
cache_dir = self.repository.paths.cache_for(probe.base)
local = Package.from_build(cache_dir, self.architecture)
local = Package.from_build(cache_dir, self.architecture, None)
packages = set(probe.packages.keys()).difference(local.packages.keys())
return list(packages)
@ -123,12 +123,14 @@ class ApplicationRepository(ApplicationProperties):
result.extend(unknown_aur(package)) # local package not found
return result
def update(self, updates: Iterable[Package]) -> Result:
def update(self, updates: Iterable[Package], packagers: Packagers | None = None) -> Result:
"""
run package updates
Args:
updates(Iterable[Package]): list of packages to update
packagers(Packagers | None, optional): optional override of username for build process
(Default value = None)
Returns:
Result: update result
@ -136,7 +138,7 @@ class ApplicationRepository(ApplicationProperties):
def process_update(paths: Iterable[Path], result: Result) -> None:
if not paths:
return # don't need to process if no update supplied
update_result = self.repository.process_update(paths)
update_result = self.repository.process_update(paths, packagers)
self.on_result(result.merge(update_result))
# process built packages
@ -148,14 +150,14 @@ class ApplicationRepository(ApplicationProperties):
tree = Tree.resolve(updates)
for num, level in enumerate(tree):
self.logger.info("processing level #%i %s", num, [package.base for package in level])
build_result = self.repository.process_build(level)
build_result = self.repository.process_build(level, packagers)
packages = self.repository.packages_built()
process_update(packages, build_result)
return build_result
def updates(self, filter_packages: Iterable[str], *,
aur: bool, local: bool, manual: bool, vcs: bool, log_fn: Callable[[str], None]) -> list[Package]:
aur: bool, local: bool, manual: bool, vcs: bool) -> list[Package]:
"""
get list of packages to run update process
@ -165,7 +167,6 @@ class ApplicationRepository(ApplicationProperties):
local(bool): enable or disable checking of local packages for updates
manual(bool): include or exclude manual updates
vcs(bool): enable or disable checking of VCS packages
log_fn(Callable[[str], None]): logger function to log updates
Returns:
list[Package]: list of out-of-dated packages
@ -179,14 +180,4 @@ class ApplicationRepository(ApplicationProperties):
if manual:
updates.update({package.base: package for package in self.repository.updates_manual()})
local_versions = {package.base: package.version for package in self.repository.packages()}
updated_packages = [package for _, package in sorted(updates.items())]
# reorder updates according to the dependency tree
tree = Tree.resolve(updated_packages)
for level in tree:
for package in level:
UpdatePrinter(package, local_versions.get(package.base)).print(
verbose=True, log_fn=log_fn, separator=" -> ")
return updated_packages
return [package for _, package in sorted(updates.items())]

View File

@ -22,6 +22,7 @@ import argparse
from ahriman.application.application import Application
from ahriman.application.handlers import Handler
from ahriman.core.configuration import Configuration
from ahriman.models.packagers import Packagers
class Add(Handler):
@ -30,8 +31,7 @@ class Add(Handler):
"""
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -40,17 +40,17 @@ class Add(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
application = Application(architecture, configuration,
report=report, unsafe=unsafe, refresh_pacman_database=args.refresh)
application = Application(architecture, configuration, report=report, refresh_pacman_database=args.refresh)
application.on_start()
application.add(args.package, args.source)
application.add(args.package, args.source, args.username)
if not args.now:
return
packages = application.updates(args.package, aur=False, local=False, manual=True, vcs=False,
log_fn=application.logger.info)
packages = application.updates(args.package, aur=False, local=False, manual=True, vcs=False)
packages = application.with_dependencies(packages, process_dependencies=args.dependencies)
result = application.update(packages)
packagers = Packagers(args.username, {package.base: package.packager for package in packages})
application.print_updates(packages, log_fn=application.logger.info)
result = application.update(packages, packagers)
Add.check_if_empty(args.exit_code, result.is_empty)

View File

@ -36,8 +36,7 @@ class Backup(Handler):
ALLOW_AUTO_ARCHITECTURE_RUN = False # it should be called only as "no-architecture"
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -46,7 +45,6 @@ class Backup(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
backup_paths = Backup.get_paths(configuration)
with TarFile(args.path, mode="w") as archive: # well we don't actually use compression

View File

@ -30,8 +30,7 @@ class Clean(Handler):
"""
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -40,9 +39,8 @@ class Clean(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
application = Application(architecture, configuration, report=report, unsafe=unsafe)
application = Application(architecture, configuration, report=report)
application.on_start()
application.clean(cache=args.cache, chroot=args.chroot, manual=args.manual, packages=args.packages,
pacman=args.pacman)

View File

@ -30,8 +30,7 @@ class Daemon(Handler):
"""
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -40,11 +39,10 @@ class Daemon(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
from ahriman.application.handlers import Update
Update.run(args, architecture, configuration, report=report, unsafe=unsafe)
Update.run(args, architecture, configuration, report=report)
timer = threading.Timer(args.interval, Daemon.run, args=[args, architecture, configuration],
kwargs={"report": report, "unsafe": unsafe})
kwargs={"report": report})
timer.start()
timer.join()

View File

@ -21,7 +21,7 @@ import argparse
from ahriman.application.handlers import Handler
from ahriman.core.configuration import Configuration
from ahriman.core.formatters import ConfigurationPrinter
from ahriman.core.formatters import ConfigurationPathsPrinter, ConfigurationPrinter, StringPrinter
class Dump(Handler):
@ -32,8 +32,7 @@ class Dump(Handler):
ALLOW_AUTO_ARCHITECTURE_RUN = False
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -42,8 +41,13 @@ class Dump(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
root, _ = configuration.check_loaded()
ConfigurationPathsPrinter(root, configuration.includes).print(verbose=True, separator=" = ")
# empty line
StringPrinter("").print(verbose=False)
dump = configuration.dump()
for section, values in sorted(dump.items()):
ConfigurationPrinter(section, values).print(verbose=not args.secure, separator=" = ")

View File

@ -97,7 +97,7 @@ class Handler:
log_handler = Log.handler(args.log_handler)
Log.load(configuration, log_handler, quiet=args.quiet, report=args.report)
with Lock(args, architecture, configuration):
cls.run(args, architecture, configuration, report=args.report, unsafe=args.unsafe)
cls.run(args, architecture, configuration, report=args.report)
return True
except ExitCode:
return False
@ -136,8 +136,7 @@ class Handler:
return 0 if all(result) else 1
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -146,7 +145,6 @@ class Handler:
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
Raises:
NotImplementedError: not implemented method

View File

@ -31,8 +31,7 @@ class Help(Handler):
ALLOW_AUTO_ARCHITECTURE_RUN = False # it should be called only as "no-architecture"
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -41,7 +40,6 @@ class Help(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
parser: argparse.ArgumentParser = args.parser()
if args.command is None:

View File

@ -32,8 +32,7 @@ class KeyImport(Handler):
ALLOW_AUTO_ARCHITECTURE_RUN = False # it should be called only as "no-architecture"
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -42,7 +41,6 @@ class KeyImport(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
application = Application(architecture, configuration, report=report, unsafe=unsafe)
application = Application(architecture, configuration, report=report)
application.repository.sign.key_import(args.key_server, args.key)

View File

@ -38,8 +38,7 @@ class Patch(Handler):
"""
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -48,9 +47,8 @@ class Patch(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
application = Application(architecture, configuration, report=report, unsafe=unsafe)
application = Application(architecture, configuration, report=report)
application.on_start()
if args.action == Action.Update and args.variable is not None:
@ -78,7 +76,7 @@ class Patch(Handler):
tuple[str, PkgbuildPatch]: package base and created PKGBUILD patch based on the diff from master HEAD
to current files
"""
package = Package.from_build(sources_dir, architecture)
package = Package.from_build(sources_dir, architecture, None)
patch = Sources.patch_create(sources_dir, *track)
return package.base, PkgbuildPatch(None, patch)

View File

@ -22,7 +22,6 @@ import argparse
from ahriman.application.application import Application
from ahriman.application.handlers import Handler
from ahriman.core.configuration import Configuration
from ahriman.core.formatters import UpdatePrinter
from ahriman.models.build_status import BuildStatusEnum
from ahriman.models.package import Package
@ -33,8 +32,7 @@ class Rebuild(Handler):
"""
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -43,9 +41,8 @@ class Rebuild(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
application = Application(architecture, configuration, report=report, unsafe=unsafe)
application = Application(architecture, configuration, report=report)
application.on_start()
packages = Rebuild.extract_packages(application, args.status, from_database=args.from_database)
@ -53,11 +50,10 @@ class Rebuild(Handler):
Rebuild.check_if_empty(args.exit_code, not updates)
if args.dry_run:
for package in updates:
UpdatePrinter(package, package.version).print(verbose=True)
application.print_updates(updates, log_fn=print)
return
result = application.update(updates)
result = application.update(updates, args.username)
Rebuild.check_if_empty(args.exit_code, result.is_empty)
@staticmethod

View File

@ -30,8 +30,7 @@ class Remove(Handler):
"""
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -40,8 +39,7 @@ class Remove(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
application = Application(architecture, configuration, report=report, unsafe=unsafe)
application = Application(architecture, configuration, report=report)
application.on_start()
application.remove(args.package)

View File

@ -31,8 +31,7 @@ class RemoveUnknown(Handler):
"""
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -41,9 +40,8 @@ class RemoveUnknown(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
application = Application(architecture, configuration, report=report, unsafe=unsafe)
application = Application(architecture, configuration, report=report)
application.on_start()
unknown_packages = application.unknown()

View File

@ -33,8 +33,7 @@ class Restore(Handler):
ALLOW_AUTO_ARCHITECTURE_RUN = False # it should be called only as "no-architecture"
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -43,7 +42,6 @@ class Restore(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
with TarFile(args.path) as archive:
archive.extractall(path=args.output)

View File

@ -47,8 +47,7 @@ class Search(Handler):
}
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -57,9 +56,8 @@ class Search(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
application = Application(architecture, configuration, report=report, unsafe=unsafe)
application = Application(architecture, configuration, report=report)
official_packages_list = Official.multisearch(*args.search, pacman=application.repository.pacman)
aur_packages_list = AUR.multisearch(*args.search, pacman=application.repository.pacman)

View File

@ -35,8 +35,7 @@ class ServiceUpdates(Handler):
ALLOW_AUTO_ARCHITECTURE_RUN = False # it should be called only as "no-architecture"
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -45,11 +44,10 @@ class ServiceUpdates(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
application = Application(architecture, configuration, report=report, unsafe=unsafe)
application = Application(architecture, configuration, report=report)
remote = Package.from_aur("ahriman", application.repository.pacman)
remote = Package.from_aur("ahriman", application.repository.pacman, None)
release = remote.version.rsplit("-", 1)[-1] # we don't store pkgrel locally, so we just append it
local_version = f"{version.__version__}-{release}"

View File

@ -26,6 +26,7 @@ from ahriman.application.application import Application
from ahriman.application.handlers import Handler
from ahriman.core.configuration import Configuration
from ahriman.models.repository_paths import RepositoryPaths
from ahriman.models.user import User
class Setup(Handler):
@ -45,8 +46,7 @@ class Setup(Handler):
SUDOERS_DIR_PATH = Path("/etc") / "sudoers.d"
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -55,12 +55,11 @@ class Setup(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
Setup.configuration_create_ahriman(args, architecture, args.repository, configuration)
configuration.reload()
application = Application(architecture, configuration, report=report, unsafe=unsafe)
application = Application(architecture, configuration, report=report)
Setup.configuration_create_makepkg(args.packager, args.makeflags_jobs, application.repository.paths)
Setup.executable_create(application.repository.paths, args.build_command, architecture)
@ -126,6 +125,9 @@ class Setup(Handler):
if args.web_unix_socket is not None:
configuration.set_option(section, "unix_socket", str(args.web_unix_socket))
if args.generate_salt:
configuration.set_option("auth", "salt", User.generate_password(20))
target = root.include / "00-setup-overrides.ini"
with target.open("w") as ahriman_configuration:
configuration.write(ahriman_configuration)
@ -175,7 +177,7 @@ class Setup(Handler):
configuration.set_option(section, "Server", mirror)
# add repository itself
configuration.set_option(repository, "SigLevel", "Optional TrustAll") # we don't care
configuration.set_option(repository, "SigLevel", "Never") # we don't care
configuration.set_option(repository, "Server", f"file://{paths.repository}")
target = source.parent / f"{prefix}-{architecture}.conf"
@ -213,7 +215,7 @@ class Setup(Handler):
"""
command = Setup.build_command(paths.root, prefix, architecture)
sudoers_file = Setup.build_command(Setup.SUDOERS_DIR_PATH, prefix, architecture)
sudoers_file.write_text(f"ahriman ALL=(ALL) NOPASSWD: {command} *\n", encoding="utf8")
sudoers_file.write_text(f"ahriman ALL=(ALL) NOPASSWD:SETENV: {command} *\n", encoding="utf8")
sudoers_file.chmod(0o400) # security!
@staticmethod

View File

@ -23,7 +23,6 @@ import sys
from pathlib import Path
from ahriman.application.application import Application
from ahriman.application.handlers import Handler
from ahriman.core.configuration import Configuration
from ahriman.core.formatters import StringPrinter
@ -37,8 +36,7 @@ class Shell(Handler):
ALLOW_MULTI_ARCHITECTURE_RUN = False
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -47,16 +45,13 @@ class Shell(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
# pylint: disable=possibly-unused-variable
application = Application(architecture, configuration, report=report, unsafe=unsafe)
if args.verbose:
# licensed by https://creativecommons.org/licenses/by-sa/3.0
path = Path(sys.prefix) / "share" / "ahriman" / "templates" / "shell"
StringPrinter(path.read_text(encoding="utf8")).print(verbose=False)
# we only want to pass application instance inside
local_variables = {"architecture": architecture, "configuration": configuration}
if args.code is None:
code.interact(local={"application": application})
code.interact(local=local_variables)
else:
code.InteractiveConsole(locals={"application": application}).runcode(args.code)
code.InteractiveConsole(locals=local_variables).runcode(args.code)

View File

@ -30,8 +30,7 @@ class Sign(Handler):
"""
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -40,6 +39,5 @@ class Sign(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
Application(architecture, configuration, report=report, unsafe=unsafe).sign(args.package)
Application(architecture, configuration, report=report).sign(args.package)

View File

@ -37,8 +37,7 @@ class Status(Handler):
ALLOW_AUTO_ARCHITECTURE_RUN = False
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -47,10 +46,9 @@ class Status(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
# we are using reporter here
client = Application(architecture, configuration, report=True, unsafe=unsafe).repository.reporter
client = Application(architecture, configuration, report=True).repository.reporter
if args.ahriman:
service_status = client.get_internal()
StatusPrinter(service_status.status).print(verbose=args.info)

View File

@ -33,8 +33,7 @@ class StatusUpdate(Handler):
ALLOW_AUTO_ARCHITECTURE_RUN = False
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -43,10 +42,9 @@ class StatusUpdate(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
# we are using reporter here
client = Application(architecture, configuration, report=True, unsafe=unsafe).repository.reporter
client = Application(architecture, configuration, report=True).repository.reporter
if args.action == Action.Update and args.package:
# update packages statuses

View File

@ -34,8 +34,7 @@ class Structure(Handler):
ALLOW_AUTO_ARCHITECTURE_RUN = False
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -44,9 +43,8 @@ class Structure(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
application = Application(architecture, configuration, report=report, unsafe=unsafe)
application = Application(architecture, configuration, report=report)
packages = application.repository.packages()
tree = Tree.resolve(packages)

View File

@ -31,8 +31,7 @@ class Triggers(Handler):
"""
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -41,9 +40,8 @@ class Triggers(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
application = Application(architecture, configuration, report=report, unsafe=unsafe)
application = Application(architecture, configuration, report=report)
if args.trigger:
loader = application.repository.triggers
loader.triggers = [loader.load_trigger(trigger, architecture, configuration) for trigger in args.trigger]

View File

@ -18,7 +18,6 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import argparse
import shlex
from ahriman.application.handlers import Handler
from ahriman.core.configuration import Configuration
@ -33,8 +32,7 @@ class UnsafeCommands(Handler):
ALLOW_AUTO_ARCHITECTURE_RUN = False # it should be called only as "no-architecture"
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -43,18 +41,17 @@ class UnsafeCommands(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
parser = args.parser()
unsafe_commands = UnsafeCommands.get_unsafe_commands(parser)
if args.command is None:
if args.command:
UnsafeCommands.check_unsafe(args.command, unsafe_commands, parser)
else:
for command in unsafe_commands:
StringPrinter(command).print(verbose=True)
else:
UnsafeCommands.check_unsafe(args.command, unsafe_commands, parser)
@staticmethod
def check_unsafe(command: str, unsafe_commands: list[str], parser: argparse.ArgumentParser) -> None:
def check_unsafe(command: list[str], unsafe_commands: list[str], parser: argparse.ArgumentParser) -> None:
"""
check if command is unsafe
@ -63,7 +60,7 @@ class UnsafeCommands(Handler):
unsafe_commands(list[str]): list of unsafe commands
parser(argparse.ArgumentParser): generated argument parser
"""
args = parser.parse_args(shlex.split(command))
args = parser.parse_args(command)
UnsafeCommands.check_if_empty(True, args.command in unsafe_commands)
@staticmethod

View File

@ -24,6 +24,7 @@ from collections.abc import Callable
from ahriman.application.application import Application
from ahriman.application.handlers import Handler
from ahriman.core.configuration import Configuration
from ahriman.models.packagers import Packagers
class Update(Handler):
@ -32,8 +33,7 @@ class Update(Handler):
"""
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -42,19 +42,19 @@ class Update(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
application = Application(architecture, configuration, report=report, unsafe=unsafe,
refresh_pacman_database=args.refresh)
application = Application(architecture, configuration, report=report, refresh_pacman_database=args.refresh)
application.on_start()
packages = application.updates(args.package, aur=args.aur, local=args.local, manual=args.manual, vcs=args.vcs,
log_fn=Update.log_fn(application, args.dry_run))
packages = application.updates(args.package, aur=args.aur, local=args.local, manual=args.manual, vcs=args.vcs)
Update.check_if_empty(args.exit_code, not packages)
if args.dry_run:
return
packages = application.with_dependencies(packages, process_dependencies=args.dependencies)
result = application.update(packages)
packagers = Packagers(args.username, {package.base: package.packager for package in packages})
application.print_updates(packages, log_fn=application.logger.info)
result = application.update(packages, packagers)
Update.check_if_empty(args.exit_code, result.is_empty)
@staticmethod

View File

@ -20,8 +20,6 @@
import argparse
import getpass
from pathlib import Path
from ahriman.application.handlers import Handler
from ahriman.core.configuration import Configuration
from ahriman.core.database import SQLite
@ -39,8 +37,7 @@ class Users(Handler):
ALLOW_AUTO_ARCHITECTURE_RUN = False # it should be called only as "no-architecture"
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -49,18 +46,13 @@ class Users(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
database = SQLite.load(configuration)
if args.action == Action.Update:
old_salt, salt = Users.get_salt(configuration)
user = Users.user_create(args)
if old_salt is None:
auth_configuration = Users.configuration_get(configuration.include)
Users.configuration_create(auth_configuration, salt, args.secure)
# if password is left blank we are not going to require salt to be set
salt = configuration.get("auth", "salt") if user.password else ""
database.user_update(user.hash_password(salt))
elif args.action == Action.List:
users = database.user_list(args.username, args.role)
@ -70,70 +62,6 @@ class Users(Handler):
elif args.action == Action.Remove:
database.user_remove(args.username)
@staticmethod
def configuration_create(configuration: Configuration, salt: str, secure: bool) -> None:
"""
enable configuration if it has been disabled
Args:
configuration(Configuration): configuration instance
salt(str): password hash salt
secure(bool): if true then set file permissions to 0o600
"""
configuration.set_option("auth", "salt", salt)
Users.configuration_write(configuration, secure)
@staticmethod
def configuration_get(include_path: Path) -> Configuration:
"""
create configuration instance
Args:
include_path(Path): path to directory with configuration includes
Returns:
Configuration: configuration instance. In case if there are local settings they will be loaded
"""
target = include_path / "00-auth.ini"
configuration = Configuration()
configuration.load(target)
configuration.architecture = "" # not user anyway
return configuration
@staticmethod
def configuration_write(configuration: Configuration, secure: bool) -> None:
"""
write configuration file
Args:
configuration(Configuration): configuration instance
secure(bool): if true then set file permissions to 0o600
"""
path, _ = configuration.check_loaded()
with path.open("w") as ahriman_configuration:
configuration.write(ahriman_configuration)
if secure:
path.chmod(0o600)
@staticmethod
def get_salt(configuration: Configuration, salt_length: int = 20) -> tuple[str | None, str]:
"""
get salt from configuration or create new string
Args:
configuration(Configuration): configuration instance
salt_length(int, optional): salt length (Default value = 20)
Returns:
tuple[str | None, str]: tuple containing salt from configuration if any and actual salt which must be
used for password hash
"""
if salt := configuration.get("auth", "salt", fallback=None):
return salt, salt
return None, User.generate_password(salt_length)
@staticmethod
def user_create(args: argparse.Namespace) -> User:
"""
@ -156,4 +84,5 @@ class Users(Handler):
if password is None:
password = read_password()
return User(username=args.username, password=password, access=args.role)
return User(username=args.username, password=password, access=args.role,
packager_id=args.packager, key=args.key)

View File

@ -39,8 +39,7 @@ class Validate(Handler):
ALLOW_AUTO_ARCHITECTURE_RUN = False
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -49,7 +48,6 @@ class Validate(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
schema = Validate.schema(architecture, configuration)
validator = Validator(configuration=configuration, schema=schema)

View File

@ -42,8 +42,7 @@ class Versions(Handler):
PEP423_PACKAGE_NAME = re.compile(r"^[A-Za-z0-9._-]+")
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -52,7 +51,6 @@ class Versions(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
VersionPrinter(f"Module version {version.__version__}",
{"Python": sys.version}).print(verbose=False, separator=" ")

View File

@ -36,8 +36,7 @@ class Web(Handler):
COMMAND_ARGS_WHITELIST = ["force", "log_handler", ""]
@classmethod
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *,
report: bool, unsafe: bool) -> None:
def run(cls, args: argparse.Namespace, architecture: str, configuration: Configuration, *, report: bool) -> None:
"""
callback for command line
@ -46,7 +45,6 @@ class Web(Handler):
architecture(str): repository architecture
configuration(Configuration): configuration instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
# we are using local import for optional dependencies
from ahriman.web.web import run_server, setup_service

View File

@ -86,6 +86,7 @@ class Lock(LazyLogging):
check if current user is actually owner of ahriman root
"""
check_user(self.paths, unsafe=self.unsafe)
self.paths.tree_create()
def clear(self) -> None:
"""
@ -116,7 +117,7 @@ class Lock(LazyLogging):
1. Check user UID
2. Check if there is lock file
3. Check web status watcher status
4. Create lock file
4. Create lock file and directory tree
5. Report to status page if enabled
Returns:

View File

@ -115,7 +115,11 @@ class AUR(Remote):
query[key] = value
try:
response = requests.get(self.DEFAULT_RPC_URL, params=query, timeout=self.DEFAULT_TIMEOUT)
response = requests.get(
self.DEFAULT_RPC_URL,
params=query,
headers={"User-Agent": self.DEFAULT_USER_AGENT},
timeout=self.DEFAULT_TIMEOUT)
response.raise_for_status()
return self.parse_response(response.json())
except requests.HTTPError as e:

View File

@ -106,6 +106,7 @@ class Official(Remote):
response = requests.get(
self.DEFAULT_RPC_URL,
params={by: args, "repo": self.DEFAULT_SEARCH_REPOSITORIES},
headers={"User-Agent": self.DEFAULT_USER_AGENT},
timeout=self.DEFAULT_TIMEOUT)
response.raise_for_status()
return self.parse_response(response.json())

View File

@ -17,6 +17,7 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from ahriman import version
from ahriman.core.alpm.pacman import Pacman
from ahriman.core.log import LazyLogging
from ahriman.models.aur_package import AURPackage
@ -26,6 +27,9 @@ class Remote(LazyLogging):
"""
base class for remote package search
Attributes:
DEFAULT_USER_AGENT(str): (class attribute) default user agent
Examples:
These classes are designed to be used without instancing. In order to achieve it several class methods are
provided: ``info``, ``multisearch`` and ``search``. Thus, the basic flow is the following::
@ -39,6 +43,8 @@ class Remote(LazyLogging):
directly, whereas ``multisearch`` splits search one by one and finds intersection between search results.
"""
DEFAULT_USER_AGENT = f"ahriman/{version.__version__}"
@classmethod
def info(cls, package_name: str, *, pacman: Pacman) -> AURPackage:
"""

View File

@ -59,12 +59,13 @@ class Task(LazyLogging):
self.makepkg_flags = configuration.getlist("build", "makepkg_flags", fallback=[])
self.makechrootpkg_flags = configuration.getlist("build", "makechrootpkg_flags", fallback=[])
def build(self, sources_dir: Path) -> list[Path]:
def build(self, sources_dir: Path, packager: str | None = None) -> list[Path]:
"""
run package build
Args:
sources_dir(Path): path to where sources are
packager(str | None, optional): optional packager override (Default value = None)
Returns:
list[Path]: paths of produced packages
@ -75,12 +76,18 @@ class Task(LazyLogging):
command.extend(["--"] + self.makepkg_flags)
self.logger.info("using %s for %s", command, self.package.base)
environment: dict[str, str] = {}
if packager is not None:
environment["PACKAGER"] = packager
self.logger.info("using environment variables %s", environment)
Task._check_output(
*command,
exception=BuildError(self.package.base),
cwd=sources_dir,
logger=self.logger,
user=self.uid)
user=self.uid,
environment=environment)
# well it is not actually correct, but we can deal with it
packages = Task._check_output(

View File

@ -38,6 +38,7 @@ class Configuration(configparser.RawConfigParser):
Required by dump and merging functions
SYSTEM_CONFIGURATION_PATH(Path): (class attribute) default system configuration path distributed by package
architecture(str | None): repository architecture
includes(list[Path]): list of includes which were read
path(Path | None): path to root configuration file
Examples:
@ -78,6 +79,7 @@ class Configuration(configparser.RawConfigParser):
})
self.architecture: str | None = None
self.path: Path | None = None
self.includes: list[Path] = []
@property
def include(self) -> Path:
@ -193,7 +195,7 @@ class Configuration(configparser.RawConfigParser):
}
# pylint and mypy are too stupid to find these methods
# pylint: disable=missing-function-docstring,multiple-statements,unused-argument
# pylint: disable=missing-function-docstring,unused-argument
def getlist(self, *args: Any, **kwargs: Any) -> list[str]: ... # type: ignore[empty-body]
def getpath(self, *args: Any, **kwargs: Any) -> Path: ... # type: ignore[empty-body]
@ -243,11 +245,13 @@ class Configuration(configparser.RawConfigParser):
"""
load configuration includes
"""
self.includes = [] # reset state
try:
for path in sorted(self.include.glob("*.ini")):
if path == self.logging_path:
continue # we don't want to load logging explicitly
self.read(path)
self.includes.append(path)
except (FileNotFoundError, configparser.NoOptionError, configparser.NoSectionError):
pass

View File

@ -191,10 +191,6 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"sign": {
"type": "dict",
"allow_unknown": True,
"keysrules": {
"type": "string",
"anyof_regex": ["^target$", "^key$", "^key_.*"],
},
"schema": {
"target": {
"type": "list",

View File

@ -0,0 +1,85 @@
#
# Copyright (c) 2021-2023 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from sqlite3 import Connection
from ahriman.core.alpm.pacman import Pacman
from ahriman.core.configuration import Configuration
from ahriman.core.util import package_like
from ahriman.models.package import Package
from ahriman.models.pacman_synchronization import PacmanSynchronization
__all__ = ["migrate_data", "steps"]
steps = [
"""
alter table users add column packager_id
""",
"""
alter table users add column key_id
""",
"""
alter table package_bases add column packager
""",
]
def migrate_data(connection: Connection, configuration: Configuration) -> None:
"""
perform data migration
Args:
connection(Connection): database connection
configuration(Configuration): configuration instance
"""
migrate_package_base_packager(connection, configuration)
def migrate_package_base_packager(connection: Connection, configuration: Configuration) -> None:
"""
migrate package packager field
Args:
connection(Connection): database connection
configuration(Configuration): configuration instance
"""
if not configuration.repository_paths.repository.is_dir():
return
_, architecture = configuration.check_loaded()
pacman = Pacman(architecture, configuration, refresh_database=PacmanSynchronization.Disabled)
package_list = []
for full_path in filter(package_like, configuration.repository_paths.repository.iterdir()):
package = Package.from_archive(full_path, pacman, remote=None)
package_list.append({
"package_base": package.base,
"packager": package.packager,
})
connection.executemany(
"""
update package_bases set
packager = :packager
where package_base = :package_base
""",
package_list
)

View File

@ -57,8 +57,9 @@ class AuthOperations(Operations):
def run(connection: Connection) -> list[User]:
return [
User(username=cursor["username"], password=cursor["password"], access=UserAccess(cursor["access"]))
for cursor in connection.execute(
User(username=row["username"], password=row["password"], access=UserAccess(row["access"]),
packager_id=row["packager_id"], key=row["key_id"])
for row in connection.execute(
"""
select * from users
where (:username is null or username = :username) and (:access is null or access = :access)
@ -91,12 +92,13 @@ class AuthOperations(Operations):
connection.execute(
"""
insert into users
(username, access, password)
(username, access, password, packager_id, key_id)
values
(:username, :access, :password)
(:username, :access, :password, :packager_id, :key_id)
on conflict (username) do update set
access = :access, password = :password
access = :access, password = :password, packager_id = :packager_id, key_id = :key_id
""",
{"username": user.username.lower(), "access": user.access.value, "password": user.password})
{"username": user.username.lower(), "access": user.access.value, "password": user.password,
"packager_id": user.packager_id, "key_id": user.key})
self.with_connection(run, commit=True)

View File

@ -76,11 +76,12 @@ class PackageOperations(Operations):
connection.execute(
"""
insert into package_bases
(package_base, version, source, branch, git_url, path, web_url)
(package_base, version, source, branch, git_url, path, web_url, packager)
values
(:package_base, :version, :source, :branch, :git_url, :path, :web_url)
(:package_base, :version, :source, :branch, :git_url, :path, :web_url, :packager)
on conflict (package_base) do update set
version = :version, branch = :branch, git_url = :git_url, path = :path, web_url = :web_url, source = :source
version = :version, branch = :branch, git_url = :git_url, path = :path, web_url = :web_url,
source = :source, packager = :packager
""",
{
"package_base": package.base,
@ -90,6 +91,7 @@ class PackageOperations(Operations):
"path": package.remote.path if package.remote is not None else None,
"web_url": package.remote.web_url if package.remote is not None else None,
"source": package.remote.source.value if package.remote is not None else None,
"packager": package.packager,
}
)
@ -163,8 +165,9 @@ class PackageOperations(Operations):
base=row["package_base"],
version=row["version"],
remote=RemoteSource.from_json(row),
packages={})
for row in connection.execute("""select * from package_bases""")
packages={},
packager=row["packager"] or None,
) for row in connection.execute("""select * from package_bases""")
}
@staticmethod

View File

@ -77,8 +77,8 @@ class PatchOperations(Operations):
"""
def run(connection: Connection) -> list[tuple[str, PkgbuildPatch]]:
return [
(cursor["package_base"], PkgbuildPatch(cursor["variable"], cursor["patch"]))
for cursor in connection.execute(
(row["package_base"], PkgbuildPatch(row["variable"], row["patch"]))
for row in connection.execute(
"""select * from patches where :package_base is null or package_base = :package_base""",
{"package_base": package_base})
]

View File

@ -22,6 +22,7 @@ from ahriman.core.formatters.string_printer import StringPrinter
from ahriman.core.formatters.aur_printer import AurPrinter
from ahriman.core.formatters.build_printer import BuildPrinter
from ahriman.core.formatters.configuration_paths_printer import ConfigurationPathsPrinter
from ahriman.core.formatters.configuration_printer import ConfigurationPrinter
from ahriman.core.formatters.package_printer import PackagePrinter
from ahriman.core.formatters.patch_printer import PatchPrinter

View File

@ -0,0 +1,52 @@
#
# Copyright (c) 2021-2023 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from pathlib import Path
from ahriman.core.formatters import StringPrinter
from ahriman.models.property import Property
class ConfigurationPathsPrinter(StringPrinter):
"""
print configuration paths
Attributes:
includes(list[Path]): list of include files
"""
def __init__(self, root: Path, includes: list[Path]) -> None:
"""
default constructor
Args:
root(Path): path to root configuration file
includes(list[Path]): list of include files
"""
StringPrinter.__init__(self, str(root))
self.includes = includes
def properties(self) -> list[Property]:
"""
convert content into printable data
Returns:
list[Property]: list of content properties
"""
return [Property("Include", str(path), is_required=True) for path in self.includes]

View File

@ -44,13 +44,13 @@ class RemotePush(LazyLogging):
remote_source(RemoteSource): repository remote source (remote pull url and branch)
"""
def __init__(self, configuration: Configuration, database: SQLite, section: str) -> None:
def __init__(self, database: SQLite, configuration: Configuration, section: str) -> None:
"""
default constructor
Args:
configuration(Configuration): configuration instance
database(SQLite): database instance
configuration(Configuration): configuration instance
section(str): settings section name
"""
self.database = database

View File

@ -105,5 +105,5 @@ class RemotePushTrigger(Trigger):
for target in self.targets:
section, _ = self.configuration.gettype(
target, self.architecture, fallback=self.CONFIGURATION_SCHEMA_FALLBACK)
runner = RemotePush(self.configuration, database, section)
runner = RemotePush(database, self.configuration, section)
runner.run(result)

View File

@ -28,6 +28,7 @@ from ahriman.core.repository.cleaner import Cleaner
from ahriman.core.util import safe_filename
from ahriman.models.package import Package
from ahriman.models.package_description import PackageDescription
from ahriman.models.packagers import Packagers
from ahriman.models.result import Result
@ -63,30 +64,35 @@ class Executor(Cleaner):
"""
raise NotImplementedError
def process_build(self, updates: Iterable[Package]) -> Result:
def process_build(self, updates: Iterable[Package], packagers: Packagers | None = None) -> Result:
"""
build packages
Args:
updates(Iterable[Package]): list of packages properties to build
packagers(Packagers | None, optional): optional override of username for build process
(Default value = None)
Returns:
Result: build result
"""
def build_single(package: Package, local_path: Path) -> None:
def build_single(package: Package, local_path: Path, packager_id: str | None) -> None:
self.reporter.set_building(package.base)
task = Task(package, self.configuration, self.paths)
task.init(local_path, self.database)
built = task.build(local_path)
built = task.build(local_path, packager_id)
for src in built:
dst = self.paths.packages / src.name
shutil.move(src, dst)
packagers = packagers or Packagers()
result = Result()
for single in updates:
with self.in_package_context(single.base), TemporaryDirectory(ignore_cleanup_errors=True) as dir_name:
try:
build_single(single, Path(dir_name))
packager = self.packager(packagers, single.base)
build_single(single, Path(dir_name), packager.packager_id)
result.add_success(single)
except Exception:
self.reporter.set_failed(single.base)
@ -158,12 +164,14 @@ class Executor(Cleaner):
return self.repo.repo_path
def process_update(self, packages: Iterable[Path]) -> Result:
def process_update(self, packages: Iterable[Path], packagers: Packagers | None = None) -> Result:
"""
sign packages, add them to repository and update repository database
Args:
packages(Iterable[Path]): list of filenames to run
packagers(Packagers | None, optional): optional override of username for build process
(Default value = None)
Returns:
Result: path to repository database
@ -176,13 +184,13 @@ class Executor(Cleaner):
shutil.move(self.paths.packages / archive.filename, self.paths.packages / safe)
archive.filename = safe
def update_single(name: str | None, package_base: str) -> None:
def update_single(name: str | None, package_base: str, packager_key: str | None) -> None:
if name is None:
self.logger.warning("received empty package name for base %s", package_base)
return # suppress type checking, it never can be none actually
# in theory, it might be NOT packages directory, but we suppose it is
full_path = self.paths.packages / name
files = self.sign.process_sign_package(full_path, package_base)
files = self.sign.process_sign_package(full_path, packager_key)
for src in files:
dst = self.paths.repository / safe_filename(src.name)
shutil.move(src, dst)
@ -192,14 +200,17 @@ class Executor(Cleaner):
current_packages = self.packages()
removed_packages: list[str] = [] # list of packages which have been removed from the base
updates = self.load_archives(packages)
packagers = packagers or Packagers()
result = Result()
for local in updates:
with self.in_package_context(local.base):
try:
packager = self.packager(packagers, local.base)
for description in local.packages.values():
rename(description, local.base)
update_single(description.filename, local.base)
update_single(description.filename, local.base, packager.key)
self.reporter.set_success(local)
result.add_success(local)

View File

@ -47,7 +47,7 @@ class Repository(Executor, UpdateHandler):
>>>
>>> configuration = Configuration()
>>> database = SQLite.load(configuration)
>>> repository = Repository.load("x86_64", configuration, database, report=True, unsafe=False)
>>> repository = Repository.load("x86_64", configuration, database, report=True)
>>> known_packages = repository.packages()
>>>
>>> build_result = repository.process_build(known_packages)
@ -58,7 +58,7 @@ class Repository(Executor, UpdateHandler):
"""
@classmethod
def load(cls, architecture: str, configuration: Configuration, database: SQLite, *, report: bool, unsafe: bool,
def load(cls, architecture: str, configuration: Configuration, database: SQLite, *, report: bool,
refresh_pacman_database: PacmanSynchronization = PacmanSynchronization.Disabled) -> Self:
"""
load instance from argument list
@ -68,7 +68,6 @@ class Repository(Executor, UpdateHandler):
configuration(Configuration): configuration instance
database(SQLite): database instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
refresh_pacman_database(PacmanSynchronization, optional): pacman database synchronization level
(Default value = PacmanSynchronization.Disabled)
@ -76,7 +75,7 @@ class Repository(Executor, UpdateHandler):
Self: fully loaded repository class instance
"""
instance = cls(architecture, configuration, database,
report=report, unsafe=unsafe, refresh_pacman_database=refresh_pacman_database)
report=report, refresh_pacman_database=refresh_pacman_database)
instance._set_context()
return instance

View File

@ -21,14 +21,15 @@ from ahriman.core.alpm.pacman import Pacman
from ahriman.core.alpm.repo import Repo
from ahriman.core.configuration import Configuration
from ahriman.core.database import SQLite
from ahriman.core.exceptions import UnsafeRunError
from ahriman.core.log import LazyLogging
from ahriman.core.sign.gpg import GPG
from ahriman.core.status.client import Client
from ahriman.core.triggers import TriggerLoader
from ahriman.core.util import check_user
from ahriman.models.packagers import Packagers
from ahriman.models.pacman_synchronization import PacmanSynchronization
from ahriman.models.repository_paths import RepositoryPaths
from ahriman.models.user import User
from ahriman.models.user_access import UserAccess
class RepositoryProperties(LazyLogging):
@ -50,7 +51,7 @@ class RepositoryProperties(LazyLogging):
vcs_allowed_age(int): maximal age of the VCS packages before they will be checked
"""
def __init__(self, architecture: str, configuration: Configuration, database: SQLite, *, report: bool, unsafe: bool,
def __init__(self, architecture: str, configuration: Configuration, database: SQLite, *, report: bool,
refresh_pacman_database: PacmanSynchronization) -> None:
"""
default constructor
@ -60,7 +61,6 @@ class RepositoryProperties(LazyLogging):
configuration(Configuration): configuration instance
database(SQLite): database instance
report(bool): force enable or disable reporting
unsafe(bool): if set no user check will be performed before path creation
refresh_pacman_database(PacmanSynchronization): pacman database synchronization level
"""
self.architecture = architecture
@ -71,11 +71,6 @@ class RepositoryProperties(LazyLogging):
self.vcs_allowed_age = configuration.getint("build", "vcs_allowed_age", fallback=0)
self.paths: RepositoryPaths = configuration.repository_paths # additional workaround for pycharm typing
try:
check_user(self.paths, unsafe=unsafe)
self.paths.tree_create()
except UnsafeRunError:
self.logger.warning("root owner differs from the current user, skipping tree creation")
self.ignore_list = configuration.getlist("build", "ignore_packages", fallback=[])
self.pacman = Pacman(architecture, configuration, refresh_database=refresh_pacman_database)
@ -83,3 +78,23 @@ class RepositoryProperties(LazyLogging):
self.repo = Repo(self.name, self.paths, self.sign.repository_sign_args)
self.reporter = Client.load(configuration, report=report)
self.triggers = TriggerLoader.load(architecture, configuration)
def packager(self, packagers: Packagers, package_base: str) -> User:
"""
extract packager from configuration having username
Args:
packagers(Packagers): packagers override holder
package_base(str): package base to lookup
Returns:
User | None: user found in database if any and empty object otherwise
"""
username = packagers.for_base(package_base)
if username is None: # none to search
return User(username="", password="", access=UserAccess.Read, packager_id=None, key=None) # nosec
if (user := self.database.user_get(username)) is not None: # found user
return user
# empty user with the username
return User(username=username, password="", access=UserAccess.Read, packager_id=None, key=None) # nosec

View File

@ -20,6 +20,7 @@
from collections.abc import Iterable
from ahriman.core.build_tools.sources import Sources
from ahriman.core.exceptions import UnknownPackageError
from ahriman.core.repository.cleaner import Cleaner
from ahriman.models.package import Package
from ahriman.models.package_source import PackageSource
@ -53,6 +54,19 @@ class UpdateHandler(Cleaner):
Returns:
list[Package]: list of packages which are out-of-dated
"""
def load_remote(package: Package) -> Package:
source = package.remote.source if package.remote is not None else None
# try to load package from base and if none found try to load by separated packages
for probe in [package.base] + sorted(package.packages.keys()):
try:
if source == PackageSource.Repository:
return Package.from_official(probe, self.pacman, None)
return Package.from_aur(probe, self.pacman, None)
except UnknownPackageError:
continue
raise UnknownPackageError(package.base)
result: list[Package] = []
for local in self.packages():
@ -61,13 +75,9 @@ class UpdateHandler(Cleaner):
continue
if filter_packages and local.base not in filter_packages:
continue
source = local.remote.source if local.remote is not None else None
try:
if source == PackageSource.Repository:
remote = Package.from_official(local.base, self.pacman)
else:
remote = Package.from_aur(local.base, self.pacman)
remote = load_remote(local)
if local.is_outdated(
remote, self.paths,
@ -98,7 +108,7 @@ class UpdateHandler(Cleaner):
with self.in_package_context(cache_dir.name):
try:
Sources.fetch(cache_dir, remote=None)
remote = Package.from_build(cache_dir, self.architecture)
remote = Package.from_build(cache_dir, self.architecture, None)
local = packages.get(remote.base)
if local is None:

View File

@ -19,7 +19,6 @@
#
import requests
from collections.abc import Generator
from pathlib import Path
from ahriman.core.configuration import Configuration
@ -165,21 +164,6 @@ class GPG(LazyLogging):
key_body = self.key_download(server, key)
GPG._check_output("gpg", "--import", input_data=key_body, logger=self.logger)
def keys(self) -> list[str]:
"""
extract list of keys described in configuration
Returns:
list[str]: list of unique keys which are set in configuration
"""
def generator() -> Generator[str, None, None]:
if self.default_key is not None:
yield self.default_key
for _, value in filter(lambda pair: pair[0].startswith("key_"), self.configuration["sign"].items()):
yield value
return sorted(set(generator()))
def process(self, path: Path, key: str) -> list[Path]:
"""
gpg command wrapper
@ -197,20 +181,21 @@ class GPG(LazyLogging):
logger=self.logger)
return [path, path.parent / f"{path.name}.sig"]
def process_sign_package(self, path: Path, package_base: str) -> list[Path]:
def process_sign_package(self, path: Path, packager_key: str | None) -> list[Path]:
"""
sign package if required by configuration
Args:
path(Path): path to file to sign
package_base(str): package base required to check for key overrides
packager_key(str | None): optional packager key to sign
Returns:
list[Path]: list of generated files including original file
"""
if SignSettings.Packages not in self.targets:
return [path]
key = self.configuration.get("sign", f"key_{package_base}", fallback=self.default_key)
key = packager_key or self.default_key
if key is None:
self.logger.error("no default key set, skip package %s sign", path)
return [path]

View File

@ -78,7 +78,7 @@ class Spawn(Thread, LazyLogging):
result = callback(args, architecture)
queue.put((process_id, result))
def _spawn_process(self, command: str, *args: str, **kwargs: str) -> None:
def _spawn_process(self, command: str, *args: str, **kwargs: str | None) -> None:
"""
spawn external ahriman process with supplied arguments
@ -94,6 +94,8 @@ class Spawn(Thread, LazyLogging):
arguments.extend(args)
# named command arguments
for argument, value in kwargs.items():
if value is None:
continue # skip null values
arguments.append(f"--{argument}")
if value:
arguments.append(value)
@ -122,27 +124,31 @@ class Spawn(Thread, LazyLogging):
kwargs = {} if server is None else {"key-server": server}
self._spawn_process("service-key-import", key, **kwargs)
def packages_add(self, packages: Iterable[str], *, now: bool) -> None:
def packages_add(self, packages: Iterable[str], username: str | None, *, now: bool) -> None:
"""
add packages
Args:
packages(Iterable[str]): packages list to add
username(str | None): optional override of username for build process
now(bool): build packages now
"""
kwargs = {"source": PackageSource.AUR.value} # avoid abusing by building non-aur packages
# avoid abusing by building non-aur packages
kwargs = {"source": PackageSource.AUR.value, "username": username}
if now:
kwargs["now"] = ""
self._spawn_process("package-add", *packages, **kwargs)
def packages_rebuild(self, depends_on: str) -> None:
def packages_rebuild(self, depends_on: str, username: str | None) -> None:
"""
rebuild packages which depend on the specified package
Args:
depends_on(str): packages dependency
username(str | None): optional override of username for build process
"""
self._spawn_process("repo-rebuild", **{"depends-on": depends_on})
kwargs = {"depends-on": depends_on, "username": username}
self._spawn_process("repo-rebuild", **kwargs)
def packages_remove(self, packages: Iterable[str]) -> None:
"""
@ -153,11 +159,15 @@ class Spawn(Thread, LazyLogging):
"""
self._spawn_process("package-remove", *packages)
def packages_update(self) -> None:
def packages_update(self, username: str | None) -> None:
"""
run full repository update
Args:
username(str | None): optional override of username for build process
"""
self._spawn_process("repo-update")
kwargs = {"username": username}
self._spawn_process("repo-update", **kwargs)
def run(self) -> None:
"""

View File

@ -53,7 +53,7 @@ class Watcher(LazyLogging):
"""
self.architecture = architecture
self.database = database
self.repository = Repository.load(architecture, configuration, database, report=False, unsafe=False)
self.repository = Repository.load(architecture, configuration, database, report=False)
self.known: dict[str, tuple[Package, BuildStatus]] = {}
self.status = BuildStatus()

View File

@ -24,6 +24,7 @@ import requests
from collections.abc import Generator
from urllib.parse import quote_plus as urlencode
from ahriman import version
from ahriman.core.configuration import Configuration
from ahriman.core.log import LazyLogging
from ahriman.core.status.client import Client
@ -140,9 +141,11 @@ class WebClient(Client, LazyLogging):
if use_unix_socket:
import requests_unixsocket # type: ignore[import]
session: requests.Session = requests_unixsocket.Session()
session.headers["User-Agent"] = f"ahriman/{version.__version__}"
return session
session = requests.Session()
session.headers["User-Agent"] = f"ahriman/{version.__version__}"
self._login(session)
return session

View File

@ -19,6 +19,7 @@
#
from ahriman.core import context
from ahriman.core.configuration import Configuration
from ahriman.core.database import SQLite
from ahriman.core.sign.gpg import GPG
from ahriman.core.support.package_creator import PackageCreator
from ahriman.core.support.pkgbuild.keyring_generator import KeyringGenerator
@ -107,8 +108,9 @@ class KeyringTrigger(Trigger):
"""
ctx = context.get()
sign = ctx.get(ContextKey("sign", GPG))
database = ctx.get(ContextKey("database", SQLite))
for target in self.targets:
generator = KeyringGenerator(sign, self.configuration, target)
generator = KeyringGenerator(database, sign, self.configuration, target)
runner = PackageCreator(self.configuration, generator)
runner.run()

View File

@ -67,5 +67,5 @@ class PackageCreator:
ctx = context.get()
database: SQLite = ctx.get(ContextKey("database", SQLite))
_, architecture = self.configuration.check_loaded()
package = Package.from_build(local_path, architecture)
package = Package.from_build(local_path, architecture, None)
database.package_update(package, BuildStatus())

View File

@ -21,6 +21,7 @@ from collections.abc import Callable
from pathlib import Path
from ahriman.core.configuration import Configuration
from ahriman.core.database import SQLite
from ahriman.core.exceptions import PkgbuildGeneratorError
from ahriman.core.sign.gpg import GPG
from ahriman.core.support.pkgbuild.pkgbuild_generator import PkgbuildGenerator
@ -42,11 +43,12 @@ class KeyringGenerator(PkgbuildGenerator):
trusted(list[str]): lif of trusted PGP keys
"""
def __init__(self, sign: GPG, configuration: Configuration, section: str) -> None:
def __init__(self, database: SQLite, sign: GPG, configuration: Configuration, section: str) -> None:
"""
default constructor
Args:
database(SQLite): database instance
sign(GPG): GPG wrapper instance
configuration(Configuration): configuration instance
section(str): settings section name
@ -55,7 +57,8 @@ class KeyringGenerator(PkgbuildGenerator):
self.name = configuration.repository_name
# configuration fields
self.packagers = configuration.getlist(section, "packagers", fallback=sign.keys())
packager_keys = [packager.key for packager in database.user_list(None, None) if packager.key is not None]
self.packagers = configuration.getlist(section, "packagers", fallback=packager_keys)
self.revoked = configuration.getlist(section, "revoked", fallback=[])
self.trusted = configuration.getlist(
section, "trusted", fallback=[sign.default_key] if sign.default_key is not None else [])
@ -148,10 +151,10 @@ class KeyringGenerator(PkgbuildGenerator):
def install(self) -> str | None:
"""
content of the install functions
content of the .install functions
Returns:
str | None: content of the install functions if any
str | None: content of the .install functions if any
"""
# copy-paste from archlinux-keyring
return f"""post_upgrade() {{

View File

@ -98,10 +98,10 @@ class PkgbuildGenerator:
def install(self) -> str | None:
"""
content of the install functions
content of the .install functions
Returns:
str | None: content of the install functions if any
str | None: content of the .install functions if any
"""
def package(self) -> str:

View File

@ -104,7 +104,7 @@ class Tree:
>>>
>>> configuration = Configuration()
>>> database = SQLite.load(configuration)
>>> repository = Repository.load("x86_64", configuration, database, report=True, unsafe=False)
>>> repository = Repository.load("x86_64", configuration, database, report=True)
>>> packages = repository.packages()
>>>
>>> tree = Tree.resolve(packages)

View File

@ -28,6 +28,7 @@ import requests
import subprocess
from collections.abc import Callable, Generator, Iterable
from dataclasses import asdict
from enum import Enum
from pathlib import Path
from pwd import getpwuid
@ -40,8 +41,10 @@ from ahriman.models.repository_paths import RepositoryPaths
__all__ = [
"check_output",
"check_user",
"dataclass_view",
"enum_values",
"exception_response_text",
"extract_user",
"filter_json",
"full_version",
"package_like",
@ -61,7 +64,8 @@ T = TypeVar("T")
def check_output(*args: str, exception: Exception | None = None, cwd: Path | None = None, input_data: str | None = None,
logger: logging.Logger | None = None, user: int | None = None) -> str:
logger: logging.Logger | None = None, user: int | None = None,
environment: dict[str, str] | None = None) -> str:
"""
subprocess wrapper
@ -73,6 +77,7 @@ def check_output(*args: str, exception: Exception | None = None, cwd: Path | Non
input_data(str | None, optional): data which will be written to command stdin (Default value = None)
logger(logging.Logger | None, optional): logger to log command result if required (Default value = None)
user(int | None, optional): run process as specified user (Default value = None)
environment(dict[str, str] | None, optional): optional environment variables if any (Default value = None)
Returns:
str: command output
@ -106,7 +111,9 @@ def check_output(*args: str, exception: Exception | None = None, cwd: Path | Non
if logger is not None:
logger.debug(single)
environment = {"HOME": getpwuid(user).pw_dir} if user is not None else {}
environment = environment or {}
if user is not None:
environment["HOME"] = getpwuid(user).pw_dir
# FIXME additional workaround for linter and type check which do not know that user arg is supported
# pylint: disable=unexpected-keyword-arg
with subprocess.Popen(args, cwd=cwd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
@ -163,6 +170,19 @@ def check_user(paths: RepositoryPaths, *, unsafe: bool) -> None:
raise UnsafeRunError(current_uid, root_uid)
def dataclass_view(instance: Any) -> dict[str, Any]:
"""
convert dataclass instance to json object
Args:
instance(Any): dataclass instance
Returns:
dict[str, Any]: json representation of the dataclass with empty field removed
"""
return asdict(instance, dict_factory=lambda fields: {key: value for key, value in fields if value is not None})
def enum_values(enum: type[Enum]) -> list[str]:
"""
generate list of enumeration values from the source
@ -190,6 +210,17 @@ def exception_response_text(exception: requests.exceptions.RequestException) ->
return result
def extract_user() -> str | None:
"""
extract user from system environment
Returns:
str | None: SUDO_USER in case if set and USER otherwise. It can return None in case if environment has been
cleared before application start
"""
return os.getenv("SUDO_USER") or os.getenv("DOAS_USER") or os.getenv("USER")
def filter_json(source: dict[str, Any], known_fields: Iterable[str]) -> dict[str, Any]:
"""
filter json object by fields used for json-to-object conversion

View File

@ -17,9 +17,10 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from dataclasses import asdict, dataclass, field
from dataclasses import dataclass, field
from typing import Any, Self
from ahriman.core.util import dataclass_view
from ahriman.models.build_status import BuildStatus
from ahriman.models.counters import Counters
@ -69,4 +70,4 @@ class InternalStatus:
Returns:
dict[str, Any]: json-friendly dictionary
"""
return asdict(self)
return dataclass_view(self)

View File

@ -23,7 +23,7 @@ from __future__ import annotations
import copy
from collections.abc import Callable, Generator, Iterable
from dataclasses import asdict, dataclass
from dataclasses import dataclass
from pathlib import Path
from pyalpm import vercmp # type: ignore[import]
from srcinfo.parse import parse_srcinfo # type: ignore[import]
@ -34,7 +34,7 @@ from ahriman.core.alpm.pacman import Pacman
from ahriman.core.alpm.remote import AUR, Official, OfficialSyncdb
from ahriman.core.exceptions import PackageInfoError
from ahriman.core.log import LazyLogging
from ahriman.core.util import check_output, full_version, srcinfo_property_list, utcnow
from ahriman.core.util import check_output, dataclass_view, full_version, srcinfo_property_list, utcnow
from ahriman.models.package_description import PackageDescription
from ahriman.models.package_source import PackageSource
from ahriman.models.remote_source import RemoteSource
@ -48,6 +48,7 @@ class Package(LazyLogging):
Attributes:
base(str): package base name
packager(str | None): package packager if available
packages(dict[str, PackageDescription): map of package names to their properties.
Filled only on load from archive
remote(RemoteSource | None): package remote source if applicable
@ -77,6 +78,7 @@ class Package(LazyLogging):
version: str
remote: RemoteSource | None
packages: dict[str, PackageDescription]
packager: str | None = None
_check_output = check_output
@ -204,16 +206,18 @@ class Package(LazyLogging):
"""
package = pacman.handle.load_pkg(str(path))
description = PackageDescription.from_package(package, path)
return cls(base=package.base, version=package.version, remote=remote, packages={package.name: description})
return cls(base=package.base, version=package.version, remote=remote, packages={package.name: description},
packager=package.packager)
@classmethod
def from_aur(cls, name: str, pacman: Pacman) -> Self:
def from_aur(cls, name: str, pacman: Pacman, packager: str | None = None) -> Self:
"""
construct package properties from AUR page
Args:
name(str): package name (either base or normal name)
pacman(Pacman): alpm wrapper instance
packager(str | None, optional): packager to be used for this build (Default value = None)
Returns:
Self: package properties
@ -224,16 +228,19 @@ class Package(LazyLogging):
base=package.package_base,
version=package.version,
remote=remote,
packages={package.name: PackageDescription.from_aur(package)})
packages={package.name: PackageDescription.from_aur(package)},
packager=packager,
)
@classmethod
def from_build(cls, path: Path, architecture: str) -> Self:
def from_build(cls, path: Path, architecture: str, packager: str | None = None) -> Self:
"""
construct package properties from sources directory
Args:
path(Path): path to package sources directory
architecture(str): load package for specific architecture
packager(str | None, optional): packager to be used for this build (Default value = None)
Returns:
Self: package properties
@ -265,7 +272,7 @@ class Package(LazyLogging):
source=PackageSource.Local,
)
return cls(base=srcinfo["pkgbase"], version=version, remote=remote, packages=packages)
return cls(base=srcinfo["pkgbase"], version=version, remote=remote, packages=packages, packager=packager)
@classmethod
def from_json(cls, dump: dict[str, Any]) -> Self:
@ -284,16 +291,18 @@ class Package(LazyLogging):
for key, value in packages_json.items()
}
remote = dump.get("remote") or {}
return cls(base=dump["base"], version=dump["version"], remote=RemoteSource.from_json(remote), packages=packages)
return cls(base=dump["base"], version=dump["version"], remote=RemoteSource.from_json(remote), packages=packages,
packager=dump.get("packager"))
@classmethod
def from_official(cls, name: str, pacman: Pacman, *, use_syncdb: bool = True) -> Self:
def from_official(cls, name: str, pacman: Pacman, packager: str | None = None, *, use_syncdb: bool = True) -> Self:
"""
construct package properties from official repository page
Args:
name(str): package name (either base or normal name)
pacman(Pacman): alpm wrapper instance
packager(str | None, optional): packager to be used for this build (Default value = None)
use_syncdb(bool, optional): use pacman databases instead of official repositories RPC (Default value = True)
Returns:
@ -305,7 +314,9 @@ class Package(LazyLogging):
base=package.package_base,
version=package.version,
remote=remote,
packages={package.name: PackageDescription.from_aur(package)})
packages={package.name: PackageDescription.from_aur(package)},
packager=packager,
)
@staticmethod
def local_files(path: Path) -> Generator[Path, None, None]:
@ -513,4 +524,4 @@ class Package(LazyLogging):
Returns:
dict[str, Any]: json-friendly dictionary
"""
return asdict(self)
return dataclass_view(self)

View File

@ -17,12 +17,12 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from dataclasses import asdict, dataclass, field, fields
from dataclasses import dataclass, field, fields
from pathlib import Path
from pyalpm import Package # type: ignore[import]
from typing import Any, Self
from ahriman.core.util import filter_json, trim_package
from ahriman.core.util import dataclass_view, filter_json, trim_package
from ahriman.models.aur_package import AURPackage
@ -172,4 +172,4 @@ class PackageDescription:
Returns:
dict[str, Any]: json-friendly dictionary
"""
return asdict(self)
return dataclass_view(self)

View File

@ -0,0 +1,46 @@
#
# Copyright (c) 2021-2023 ahriman team.
#
# This file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from dataclasses import dataclass, field
@dataclass(frozen=True)
class Packagers:
"""
holder for packagers overrides
Attributes:
default(str | None): default packager username if any to be used if no override for the specified base was found
overrides: dict[str, str | None]: packager username override for specific package base
"""
default: str | None = None
overrides: dict[str, str | None] = field(default_factory=dict)
def for_base(self, package_base: str) -> str | None:
"""
extract username for the specified package base
Args:
package_base(str): package base to lookup
Returns:
str | None: package base override if set and default packager username otherwise
"""
return self.overrides.get(package_base) or self.default

View File

@ -17,11 +17,11 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from dataclasses import asdict, dataclass, fields
from dataclasses import dataclass, fields
from pathlib import Path
from typing import Any, Self
from ahriman.core.util import filter_json
from ahriman.core.util import dataclass_view, filter_json
from ahriman.models.package_source import PackageSource
@ -118,4 +118,4 @@ class RemoteSource:
Returns:
dict[str, Any]: json-friendly dictionary
"""
return asdict(self)
return dataclass_view(self)

View File

@ -34,12 +34,14 @@ class User:
username(str): username
password(str): hashed user password with salt
access(UserAccess): user role
packager_id(str | None): packager id to be used. If not set, the default service packager will be used
key(str | None): personal packager key if any. If user id is empty, it is interpreted as default key
Examples:
Simply create user from database data and perform required validation::
>>> password = User.generate_password(24)
>>> user = User("ahriman", password, UserAccess.Full)
>>> user = User(username="ahriman", password=password, access=UserAccess.Full, packager_id=None, key=None)
Since the password supplied may be plain text, the ``hash_password`` method can be used to hash the password::
@ -61,9 +63,18 @@ class User:
username: str
password: str
access: UserAccess
packager_id: str | None
key: str | None
_HASHER = sha512_crypt
def __post_init__(self) -> None:
"""
remove empty fields
"""
object.__setattr__(self, "packager_id", self.packager_id or None)
object.__setattr__(self, "key", self.key or None)
@classmethod
def from_option(cls, username: str | None, password: str | None,
access: UserAccess = UserAccess.Read) -> Self | None:
@ -80,7 +91,7 @@ class User:
"""
if username is None or password is None:
return None
return cls(username=username, password=password, access=access)
return cls(username=username, password=password, access=access, packager_id=None, key=None)
@staticmethod
def generate_password(length: int) -> str:
@ -149,4 +160,4 @@ class User:
Returns:
str: unique string representation
"""
return f"User(username={self.username}, access={self.access})"
return f"User(username={self.username}, access={self.access}, packager_id={self.packager_id}, key={self.key})"

View File

@ -17,4 +17,4 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
__version__ = "2.9.0"
__version__ = "2.10.1"

View File

@ -48,9 +48,9 @@ def _info() -> dict[str, Any]:
* VCS packages support.
* Official repository support.
* Ability to patch AUR packages and even create package from local PKGBUILDs.
* Sign support with gpg (repository, package, per package settings).
* Sign support with gpg (repository, package), multiple packagers support.
* Triggers for repository updates, e.g. synchronization to remote services (rsync, s3 and github) and report generation (email, html, telegram).
* Repository status interface with optional authorization and control options
* Repository status interface with optional authorization and control options.
<security-definitions />
""",

View File

@ -148,7 +148,7 @@ def setup_auth(application: Application, configuration: Configuration, validator
setup_session(application, storage)
authorization_policy = _AuthorizationPolicy(validator)
identity_policy = aiohttp_security.SessionIdentityPolicy()
identity_policy = application["identity"] = aiohttp_security.SessionIdentityPolicy()
aiohttp_security.setup(application, identity_policy, authorization_policy)
application.middlewares.append(_auth_handler(validator.allow_read_only))

View File

@ -17,3 +17,22 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from ahriman.web.schemas.aur_package_schema import AURPackageSchema
from ahriman.web.schemas.auth_schema import AuthSchema
from ahriman.web.schemas.counters_schema import CountersSchema
from ahriman.web.schemas.error_schema import ErrorSchema
from ahriman.web.schemas.internal_status_schema import InternalStatusSchema
from ahriman.web.schemas.log_schema import LogSchema
from ahriman.web.schemas.login_schema import LoginSchema
from ahriman.web.schemas.logs_schema import LogsSchema
from ahriman.web.schemas.oauth2_schema import OAuth2Schema
from ahriman.web.schemas.package_name_schema import PackageNameSchema
from ahriman.web.schemas.package_names_schema import PackageNamesSchema
from ahriman.web.schemas.package_properties_schema import PackagePropertiesSchema
from ahriman.web.schemas.package_schema import PackageSchema
from ahriman.web.schemas.package_status_schema import PackageStatusSimplifiedSchema, PackageStatusSchema
from ahriman.web.schemas.pgp_key_id_schema import PGPKeyIdSchema
from ahriman.web.schemas.pgp_key_schema import PGPKeySchema
from ahriman.web.schemas.remote_schema import RemoteSchema
from ahriman.web.schemas.search_schema import SearchSchema
from ahriman.web.schemas.status_schema import StatusSchema

View File

@ -44,3 +44,7 @@ class PackageSchema(Schema):
keys=fields.String(), values=fields.Nested(PackagePropertiesSchema()), required=True, metadata={
"description": "Packages which belong to this base",
})
packager = fields.String(metadata={
"description": "packager for the last success package build",
"example": "John Doe <john@doe.com>",
})

View File

@ -183,3 +183,16 @@ class BaseView(View, CorsViewMixin):
return response
self._raise_allowed_methods()
async def username(self) -> str | None:
"""
extract username from request if any
Returns:
str | None: authorized username if any and None otherwise (e.g. if authorization is disabled)
"""
policy = self.request.app.get("identity")
if policy is not None:
identity: str = await policy.identify(self.request)
return identity
return None

View File

@ -22,9 +22,7 @@ import aiohttp_apispec # type: ignore[import]
from aiohttp.web import HTTPBadRequest, HTTPNoContent
from ahriman.models.user_access import UserAccess
from ahriman.web.schemas.auth_schema import AuthSchema
from ahriman.web.schemas.error_schema import ErrorSchema
from ahriman.web.schemas.package_names_schema import PackageNamesSchema
from ahriman.web.schemas import AuthSchema, ErrorSchema, PackageNamesSchema
from ahriman.web.views.base import BaseView
@ -67,6 +65,7 @@ class AddView(BaseView):
except Exception as e:
raise HTTPBadRequest(reason=str(e))
self.spawner.packages_add(packages, now=True)
username = await self.username()
self.spawner.packages_add(packages, username, now=True)
raise HTTPNoContent()

Some files were not shown because too many files have changed in this diff Show More