mirror of
https://github.com/arcan1s/ahriman.git
synced 2025-04-24 15:27:17 +00:00
docs: docs review
This commit is contained in:
parent
c130f52163
commit
a872ecfc23
@ -12,7 +12,7 @@ Depending on the goal the package can be used in different ways. Nevertheless, i
|
||||
from ahriman.models.repository_id import RepositoryId
|
||||
|
||||
repository_id = RepositoryId("x86_64", "aur-clone")
|
||||
configuration = Configuration.from_path(Path("/etc/ahriman.ini"), architecture)
|
||||
configuration = Configuration.from_path(Path("/etc/ahriman.ini"), repository_id)
|
||||
database = SQLite.load(configuration)
|
||||
|
||||
At this point there are ``configuration`` and ``database`` instances which can be used later at any time anywhere, e.g.
|
||||
|
||||
@ -6,7 +6,7 @@ Package structure
|
||||
|
||||
Packages have strict rules of importing:
|
||||
|
||||
* ``ahriman.application`` package must not be used anywhere except for itself.
|
||||
* ``ahriman.application`` package must not be used outside of this package.
|
||||
* ``ahriman.core`` and ``ahriman.models`` packages don't have any import restriction. Actually we would like to totally restrict importing of ``core`` package from ``models``, but it is impossible at the moment.
|
||||
* ``ahriman.web`` package is allowed to be imported from ``ahriman.application`` (web handler only, only ``ahriman.web.web`` methods). It also must not be imported globally, only local import is allowed.
|
||||
|
||||
@ -19,10 +19,12 @@ Full dependency diagram:
|
||||
``ahriman.application`` package
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
This package contains application (aka executable) related classes and everything for it. It also contains package called ``ahriman.application.handlers`` in which all available subcommands are described as separated classes derived from base ``ahriman.application.handlers.Handler`` class.
|
||||
This package contains application (aka executable) related classes and everything for it. It also contains package called ``ahriman.application.handlers`` in which all available subcommands are described as separated classes derived from the base ``ahriman.application.handlers.Handler`` class.
|
||||
|
||||
``ahriman.application.application.Application`` (god class) is used for any interaction from parsers with repository. It is divided into multiple traits by functions (package related and repository related) in the same package.
|
||||
|
||||
``ahriman.application.application.workers`` package contains specific wrappers for local and remote build processes.
|
||||
|
||||
``ahriman.application.ahriman`` contains only command line parses and executes specified ``Handler`` on success, ``ahriman.application.lock.Lock`` is additional class which provides file-based lock and also performs some common checks.
|
||||
|
||||
``ahriman.core`` package
|
||||
@ -31,13 +33,13 @@ This package contains application (aka executable) related classes and everythin
|
||||
This package contains everything required for the most of application actions and it is separated into several packages:
|
||||
|
||||
* ``ahriman.core.alpm`` package controls pacman related functions. It provides wrappers for ``pyalpm`` library and safe calls for repository tools (``repo-add`` and ``repo-remove``). Also this package contains ``ahriman.core.alpm.remote`` package which provides wrapper for remote sources (e.g. AUR RPC and official repositories RPC).
|
||||
* ``ahriman.core.auth`` package provides classes for authorization methods used by web mostly. Base class is ``ahriman.core.auth.Auth`` which must be called by ``load`` method.
|
||||
* ``ahriman.core.auth`` package provides classes for authorization methods used by web mostly. Base class is ``ahriman.core.auth.Auth`` which must be instantiated by ``load`` method.
|
||||
* ``ahriman.core.build_tools`` is a package which provides wrapper for ``devtools`` commands.
|
||||
* ``ahriman.core.configuration`` contains extension for standard ``configparser`` library and some validation related classes.
|
||||
* ``ahriman.core.database`` is everything including data and schema migrations for database.
|
||||
* ``ahriman.core.database`` is everything for database, including data and schema migrations.
|
||||
* ``ahriman.core.formatters`` package provides ``Printer`` sub-classes for printing data (e.g. package properties) to stdout which are used by some handlers.
|
||||
* ``ahriman.core.gitremote`` is a package with remote PKGBUILD triggers. Should not be called directly.
|
||||
* ``ahriman.core.http`` package provides HTTP clients which can be later used by other classes.
|
||||
* ``ahriman.core.http`` package provides HTTP clients which can be used later by other classes.
|
||||
* ``ahriman.core.log`` is a log utils package. It includes logger loader class, custom HTTP based logger and access logger for HTTP services with additional filters.
|
||||
* ``ahriman.core.report`` is a package with reporting triggers. Should not be called directly.
|
||||
* ``ahriman.core.repository`` contains several traits and base repository (``ahriman.core.repository.Repository`` class) implementation.
|
||||
@ -105,7 +107,7 @@ For historical reasons and in order to keep backward compatibility some subcomma
|
||||
Filesystem tree
|
||||
---------------
|
||||
|
||||
The application supports two types of trees, one is for the legacy configuration (when there were no repository name explicit configuration available) and another one is the new-style tree. This document describes only new-style tree in order to avoid deprecated structures.
|
||||
The application supports two types of trees, one is for the legacy configuration (when there were no explicit repository name configuration available) and another one is the new-style tree. This document describes only new-style tree in order to avoid deprecated structures.
|
||||
|
||||
Having default root as ``/var/lib/ahriman`` (differs from container though), the directory structure is the following:
|
||||
|
||||
@ -141,7 +143,7 @@ There are multiple subdirectories, some of them are commons for any repository,
|
||||
|
||||
* ``cache`` is a directory with locally stored PKGBUILD's and VCS packages. It is common for all repositories and architectures.
|
||||
* ``chroot/{repository}`` is a chroot directory for ``devtools``. It is specific for each repository, but shared for different architectures inside (the ``devtools`` handles architectures automatically).
|
||||
* ``packages/{repository}/{architecture}`` is a directory with prebuilt packages. When package is built, first it will be uploaded to this directory and later will be handled by update process. It is architecture and repository specific.
|
||||
* ``packages/{repository}/{architecture}`` is a directory with prebuilt packages. When a package is built, first it will be uploaded to this directory and later will be handled by update process. It is architecture and repository specific.
|
||||
* ``pacman/{repository}/{architecture}`` is the repository and architecture specific caches for pacman's databases.
|
||||
* ``repository/{repository}/{architecture}`` is a repository packages directory.
|
||||
|
||||
@ -155,16 +157,16 @@ The service uses SQLite database in order to store some internal info.
|
||||
Database instance
|
||||
^^^^^^^^^^^^^^^^^
|
||||
|
||||
All methods related to specific part of database (basically operations per table) are split into different traits located inside ``ahriman.core.database.operations`` package. The base trait ``ahriman.core.database.operations.Operations`` also provides generic methods for database access (e.g. row converters and transactional support).
|
||||
All methods related to the specific part of database (basically operations per table) are split into different traits located inside ``ahriman.core.database.operations`` package. The base trait ``ahriman.core.database.operations.Operations`` also provides generic methods for database access (e.g. row converters and transactional support).
|
||||
|
||||
The ``ahriman.core.database.SQLite`` class itself derives from all of these traits and implements methods for initialization, including migrations.
|
||||
|
||||
Schema and data migrations
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
The schema migration are applied according to current ``pragma user_info`` values, located at ``ahriman.core.database.migrations`` package and named as ``m000_migration_name.py`` (the preceding ``m`` is required in order to import migration content for tests). Additional class ``ahriman.core.database.migrations.Migrations`` reads all migrations automatically and applies them in alphabetical order.
|
||||
The schema migrations are applied according to current ``pragma user_info`` values, located at ``ahriman.core.database.migrations`` package and named as ``m000_migration_name.py`` (the preceding ``m`` is required in order to import migration content for tests). Additional class ``ahriman.core.database.migrations.Migrations`` reads all migrations automatically and applies them in alphabetical order.
|
||||
|
||||
These migrations can also contain data migrations. Though the recommended way is to migrate data directly from SQL requests, sometimes it is required to have external data (like packages list) in order to set correct data. To do so, special method ``migrate_data`` is used.
|
||||
These migrations can also contain data migrations. Though the recommended way is to migrate data directly from SQL queries, sometimes it is required to have external data (like packages list) in order to set correct data. To do so, special method ``migrate_data`` is used.
|
||||
|
||||
Type conversions
|
||||
^^^^^^^^^^^^^^^^
|
||||
@ -180,15 +182,15 @@ By default package build operations are performed with ``PACKAGER`` which is spe
|
||||
|
||||
* If packager is not set, it reads environment variables (e.g. ``SUDO_USER`` and ``USER``), otherwise it uses value from command line.
|
||||
* It checks users for the specified username and tries to extract packager variable from it.
|
||||
* If packager value has been found, it will be passed as ``PACKAGER`` system variable (sudo configuration required).
|
||||
* If packager value has been found, it will be passed as ``PACKAGER`` system variable (additional sudo configuration might be required).
|
||||
|
||||
Add new packages or rebuild existing
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Idea is to copy package to the directory from which it will be handled at the next update run. Different variants are supported:
|
||||
Idea is to add package to a build queue from which it will be handled automatically during the next update run. Different variants are supported:
|
||||
|
||||
* If supplied argument is file then application moves the file to the directory with built packages. Same rule applies for directory, but in this case it copies every package-like file from the specified directory.
|
||||
* If supplied argument is directory and there is ``PKGBUILD`` file there it will be treated as local package. In this case it will queue this package to build and copy source files (``PKGBUILD`` and ``.SRCINFO``) to caches.
|
||||
* If supplied argument is file, then application moves the file to the directory with built packages. Same rule applies for directory, but in this case it copies every package-like file from the specified directory.
|
||||
* If supplied argument is directory and there is ``PKGBUILD`` file there, it will be treated as local package. In this case it will queue this package to build and copy source files (``PKGBUILD`` and ``.SRCINFO``) to caches.
|
||||
* If supplied argument is not file then application tries to lookup for the specified name in AUR and clones it into the directory with manual updates. This scenario can also handle package dependencies which are missing in repositories.
|
||||
|
||||
This logic can be overwritten by specifying the ``source`` parameter, which is partially useful if you would like to add package from AUR, but there is local directory cloned from AUR. Also official repositories calls are hidden behind explicit source definition.
|
||||
@ -206,7 +208,7 @@ This flow removes package from filesystem, updates repository database and also
|
||||
Update packages
|
||||
^^^^^^^^^^^^^^^
|
||||
|
||||
This feature is divided into to stages: check AUR for updates and run rebuild for required packages. Whereas check does not do anything except for check itself, update flow is the following:
|
||||
This feature is divided into to the following stages: check AUR for updates and run rebuild for required packages. Whereas check does not do anything except for check itself, update flow is the following:
|
||||
|
||||
#. Process every built package first. Those packages are usually added manually.
|
||||
#. Run sync and report methods.
|
||||
@ -222,13 +224,33 @@ This feature is divided into to stages: check AUR for updates and run rebuild fo
|
||||
|
||||
After any step any package data is being removed.
|
||||
|
||||
In case if there are configured workers, the build process itself will be delegated to the remote instances. Packages will be partitioned to the chunks according to the amount of configured workers.
|
||||
|
||||
Distributed builds
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
This feature consists of two parts:
|
||||
|
||||
* Upload built packages to the node.
|
||||
* Delegate packages building to separated nodes.
|
||||
|
||||
The upload process is performed via special API endpoint, which is disabled by default, and is performed in several steps:
|
||||
|
||||
#. Upload package to temporary file.
|
||||
#. Copy content from temporary file to the built package directory with dot (``.``) prefix.
|
||||
#. Rename copied file, removing preceding dot.
|
||||
|
||||
After success upload, the update process must be called as usual in order to copy built packages to the main repository tree.
|
||||
|
||||
On the other side, the delegation uses upload feature, but in addition it also calls external services in order to trigger build process. The packages are separated to chunks based on the amount of the configured workers and their dependencies.
|
||||
|
||||
pkgrel bump rules
|
||||
^^^^^^^^^^^^^^^^^
|
||||
|
||||
The application is able to automatically bump package release (``pkgrel``) during build process if there is duplicate version in repository. The version will be incremented as following:
|
||||
|
||||
#. Get version of the remote package.
|
||||
#. Get version of the local package if any.
|
||||
#. Get version of the local package if available.
|
||||
#. If local version is not set, proceed with remote one.
|
||||
#. If local version is set and epoch or package version (``pkgver``) are different, proceed with remote version.
|
||||
#. If local version is set and remote version is newer than local one, proceed with remote.
|
||||
@ -247,7 +269,7 @@ Configuration
|
||||
Enumerations
|
||||
^^^^^^^^^^^^
|
||||
|
||||
All enumerations are derived from ``str`` and ``enum.Enum``. Integer enumerations in general are not allowed, because most of operations require conversions from string variable. Derivation from string class is required to make json conversions implicitly (e.g. during calling ``json.dumps`` methods).
|
||||
All enumerations are derived from ``enum.StrEnum``. Integer enumerations in general are not allowed, because most of operations require conversions from string variable. Derivation from string based enumeration is required to make json conversions implicitly (e.g. during calling ``json.dumps`` methods).
|
||||
|
||||
In addition, some enumerations provide ``from_option`` class methods in order to allow some flexibility while reading configuration options.
|
||||
|
||||
@ -319,9 +341,9 @@ There are several supported synchronization providers, currently they are ``rsyn
|
||||
|
||||
``rsync`` provider does not have any specific logic except for running external rsync application with configured arguments. The service does not handle SSH configuration, thus it has to be configured before running application manually.
|
||||
|
||||
``s3`` provider uses ``boto3`` package and implements sync feature. The files are stored in architecture directory (e.g. if bucket is ``repository``, packages will be stored in ``repository/aur-clone/x86_64`` for the ``aur-clone`` repository ``x86_64`` architecture), bucket must be created before any action and API key must have permissions to write to the bucket. No external configuration required. In order to upload only changed files the service compares calculated hashes with the Amazon ETags, used realization is described `here <https://teppen.io/2018/10/23/aws_s3_verify_etags/>`__.
|
||||
``s3`` provider uses ``boto3`` package and implements sync feature. The files are stored in architecture specific directory (e.g. if bucket is ``repository``, packages will be stored in ``repository/aur-clone/x86_64`` for the ``aur-clone`` repository and ``x86_64`` architecture), bucket must be created before any action and API key must have permissions to write to the bucket. No external configuration required. In order to upload only changed files the service compares calculated hashes with the Amazon ETags, the implementation used is described `here <https://teppen.io/2018/10/23/aws_s3_verify_etags/>`__.
|
||||
|
||||
``github`` provider authenticates through basic auth, API key with repository write permissions is required. There will be created a release with the name of the architecture in case if it does not exist; files will be uploaded to the release assets. It also stores array of files and their MD5 checksums in release body in order to upload only changed ones. According to the Github API in case if there is already uploaded asset with the same name (e.g. database files), asset will be removed first.
|
||||
``github`` provider authenticates through basic auth, API key with repository write permissions is required. There will be created a release with the name of the architecture in case if it does not exist; files will be uploaded to the release assets. It also stores array of files and their MD5 checksums in release body in order to upload only changed ones. According to the GitHub API in case if there is already uploaded asset with the same name (e.g. database files), asset will be removed first.
|
||||
|
||||
Additional features
|
||||
^^^^^^^^^^^^^^^^^^^
|
||||
@ -340,7 +362,7 @@ Web application
|
||||
Web application requires the following python packages to be installed:
|
||||
|
||||
* Core part requires ``aiohttp`` (application itself), ``aiohttp_jinja2`` and ``Jinja2`` (HTML generation from templates).
|
||||
* Additional web features also require ``aiohttp-apispec`` (autogenerated documentation), ``aiohttp_cors`` (CORS support, required by documentation)
|
||||
* Additional web features also require ``aiohttp-apispec`` (autogenerated documentation), ``aiohttp_cors`` (CORS support, required by documentation).
|
||||
* In addition, ``aiohttp_debugtoolbar`` is required for debug panel. Please note that this option does not work together with authorization and basically must not be used in production.
|
||||
* In addition, authorization feature requires ``aiohttp_security``, ``aiohttp_session`` and ``cryptography``.
|
||||
* In addition to base authorization dependencies, OAuth2 also requires ``aioauth-client`` library.
|
||||
@ -363,7 +385,7 @@ Web views
|
||||
|
||||
All web views are defined in separated package and derived from ``ahriman.web.views.base.Base`` class which provides typed interfaces for web application.
|
||||
|
||||
REST API supports both form and JSON data, but the last one is recommended.
|
||||
REST API supports only JSON data.
|
||||
|
||||
Different APIs are separated into different packages:
|
||||
|
||||
@ -377,12 +399,12 @@ The views are also divided by supporting API versions (e.g. ``v1``, ``v2``).
|
||||
Templating
|
||||
^^^^^^^^^^
|
||||
|
||||
Package provides base jinja templates which can be overridden by settings. Vanilla templates are actively using bootstrap library.
|
||||
Package provides base jinja templates which can be overridden by settings. Vanilla templates actively use bootstrap library.
|
||||
|
||||
Requests and scopes
|
||||
^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Service provides optional authorization which can be turned on in settings. In order to control user access there are two levels of authorization - read-only (only GET-like requests) and write (anything) which are provided by each web view directly.
|
||||
Service provides optional authorization which can be turned on in settings. In order to control user access there are two levels of authorization - read-only (only GET-like requests) and write (anything), settings for which are provided by each web view directly.
|
||||
|
||||
If this feature is configured any request will be prohibited without authentication. In addition, configuration flag ``auth.allow_read_only`` can be used in order to allow read-only operations - reading index page and packages - without authorization.
|
||||
|
||||
@ -393,4 +415,4 @@ External calls
|
||||
|
||||
Web application provides external calls to control main service. It spawns child process with specific arguments and waits for its termination. This feature must be used either with authorization or in safe (i.e. when status page is not available world-wide) environment.
|
||||
|
||||
For most actions it also extracts user from authentication (if provided) and passes it to underlying process.
|
||||
For most actions it also extracts user from authentication (if provided) and passes it to the underlying process.
|
||||
|
||||
@ -39,7 +39,7 @@ It will check current settings on common errors and compare configuration with k
|
||||
|
||||
Base configuration settings.
|
||||
|
||||
* ``apply_migrations`` - perform migrations on application start, boolean, optional, default ``yes``. Useful if you are using git version. Note, however, that this option must be changed only if you know what to do and going to handle migrations automatically.
|
||||
* ``apply_migrations`` - perform migrations on application start, boolean, optional, default ``yes``. Useful if you are using git version. Note, however, that this option must be changed only if you know what to do and going to handle migrations manually.
|
||||
* ``database`` - path to SQLite database, string, required.
|
||||
* ``include`` - path to directory with configuration files overrides, string, optional.
|
||||
* ``logging`` - path to logging configuration, string, required. Check ``logging.ini`` for reference.
|
||||
@ -50,7 +50,7 @@ Base configuration settings.
|
||||
libalpm and AUR related configuration. Group name can refer to architecture, e.g. ``alpm:x86_64`` can be used for x86_64 architecture specific settings.
|
||||
|
||||
* ``database`` - path to pacman system database cache, string, required.
|
||||
* ``mirror`` - package database mirror used by pacman for syncronization, string, required. This option supports standard pacman substitutions with ``$arch`` and ``$repo``. Note that the mentioned mirror should contain all repositories which are set by ``alpm.repositories`` option.
|
||||
* ``mirror`` - package database mirror used by pacman for synchronization, string, required. This option supports standard pacman substitutions with ``$arch`` and ``$repo``. Note that the mentioned mirror should contain all repositories which are set by ``alpm.repositories`` option.
|
||||
* ``repositories`` - list of pacman repositories, space separated list of strings, required.
|
||||
* ``root`` - root for alpm library, string, required.
|
||||
* ``use_ahriman_cache`` - use local pacman package cache instead of system one, boolean, required. With this option enabled you might want to refresh database periodically (available as additional flag for some subcommands).
|
||||
@ -64,14 +64,14 @@ Base authorization settings. ``OAuth`` provider requires ``aioauth-client`` libr
|
||||
* ``allow_read_only`` - allow requesting status APIs without authorization, boolean, required.
|
||||
* ``client_id`` - OAuth2 application client ID, string, required in case if ``oauth`` is used.
|
||||
* ``client_secret`` - OAuth2 application client secret key, string, required in case if ``oauth`` is used.
|
||||
* ``cookie_secret_key`` - secret key which will be used for cookies encryption, string, optional. It must be 32 url-safe base64-encoded bytes and can be generated as following ``base64.urlsafe_b64encode(os.urandom(32)).decode("utf8")``. If not set, it will be generated automatically; note, however, that in this case, all sessions will be automatically expired during restart.
|
||||
* ``max_age`` - parameter which controls both cookie expiration and token expiration inside the service, integer, optional, default is 7 days.
|
||||
* ``cookie_secret_key`` - secret key which will be used for cookies encryption, string, optional. It must be 32 url-safe base64-encoded bytes and can be generated as following ``base64.urlsafe_b64encode(os.urandom(32)).decode("utf8")``. If not set, it will be generated automatically; note, however, that in this case, all sessions will be automatically invalidated during the service restart.
|
||||
* ``max_age`` - parameter which controls both cookie expiration and token expiration inside the service in seconds, integer, optional, default is 7 days.
|
||||
* ``oauth_icon`` - OAuth2 login button icon, string, optional, default is ``google``. Must be valid `Bootstrap icon <https://icons.getbootstrap.com/>`__ name.
|
||||
* ``oauth_provider`` - OAuth2 provider class name as is in ``aioauth-client`` (e.g. ``GoogleClient``, ``GithubClient`` etc), string, required in case if ``oauth`` is used.
|
||||
* ``oauth_scopes`` - scopes list for OAuth2 provider, which will allow retrieving user email (which is used for checking user permissions), e.g. ``https://www.googleapis.com/auth/userinfo.email`` for ``GoogleClient`` or ``user:email`` for ``GithubClient``, space separated list of strings, required in case if ``oauth`` is used.
|
||||
* ``salt`` - additional password hash salt, string, optional.
|
||||
|
||||
Authorized users are stored inside internal database, if any of external provides are used the password field for non-service users must be empty.
|
||||
Authorized users are stored inside internal database, if any of external providers (e.g. ``oauth``) are used, the password field for non-service users must be empty.
|
||||
|
||||
``build:*`` groups
|
||||
------------------
|
||||
@ -83,9 +83,9 @@ Build related configuration. Group name can refer to architecture, e.g. ``build:
|
||||
* ``ignore_packages`` - list packages to ignore during a regular update (manual update will still work), space separated list of strings, optional.
|
||||
* ``makepkg_flags`` - additional flags passed to ``makepkg`` command, space separated list of strings, optional.
|
||||
* ``makechrootpkg_flags`` - additional flags passed to ``makechrootpkg`` command, space separated list of strings, optional.
|
||||
* ``triggers`` - list of ``ahriman.core.triggers.Trigger`` class implementation (e.g. ``ahriman.core.report.ReportTrigger ahriman.core.upload.UploadTrigger``) which will be loaded and run at the end of processing, space separated list of strings, optional. You can also specify triggers by their paths, e.g. ``/usr/lib/python3.10/site-packages/ahriman/core/report/report.py.ReportTrigger``. Triggers are run in the order of mention.
|
||||
* ``triggers`` - list of ``ahriman.core.triggers.Trigger`` class implementation (e.g. ``ahriman.core.report.ReportTrigger ahriman.core.upload.UploadTrigger``) which will be loaded and run at the end of processing, space separated list of strings, optional. You can also specify triggers by their paths, e.g. ``/usr/lib/python3.10/site-packages/ahriman/core/report/report.py.ReportTrigger``. Triggers are run in the order of definition.
|
||||
* ``triggers_known`` - optional list of ``ahriman.core.triggers.Trigger`` class implementations which are not run automatically and used only for trigger discovery and configuration validation.
|
||||
* ``vcs_allowed_age`` - maximal age in seconds of the VCS packages before their version will be updated with its remote source, integer, optional, default ``604800``.
|
||||
* ``vcs_allowed_age`` - maximal age in seconds of the VCS packages before their version will be updated with its remote source, integer, optional, default is 7 days.
|
||||
* ``workers`` - list of worker nodes addresses used for build process, space separated list of strings, optional. Each worker address must be valid and reachable url, e.g. ``https://10.0.0.1:8080``. If none set, the build process will be run on the current node.
|
||||
|
||||
``repository`` group
|
||||
@ -118,7 +118,7 @@ Reporting to web service related settings. In most cases there is fallback to we
|
||||
``web`` group
|
||||
-------------
|
||||
|
||||
Web server settings. If any of ``host``/``port`` is not set, web integration will be disabled. This feature requires ``aiohttp`` libraries to be installed.
|
||||
Web server settings. This feature requires ``aiohttp`` libraries to be installed.
|
||||
|
||||
* ``address`` - optional address in form ``proto://host:port`` (``port`` can be omitted in case of default ``proto`` ports), will be used instead of ``http://{host}:{port}`` in case if set, string, optional. This option is required in case if ``OAuth`` provider is used.
|
||||
* ``debug`` - enable debug toolbar, boolean, optional, default ``no``.
|
||||
|
||||
85
docs/faq.rst
85
docs/faq.rst
@ -76,7 +76,7 @@ the ``extra-i686-build`` command will be used for ``i686`` architecture. You can
|
||||
How to generate build reports
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Normally you probably like to generate only one report for the specific type, e.g. only one email report. In order to do it you will need to have the following configuration:
|
||||
Normally you would probably like to generate only one report for the specific type, e.g. only one email report. In order to do so you will need to have the following configuration:
|
||||
|
||||
.. code-block:: ini
|
||||
|
||||
@ -155,7 +155,7 @@ For that purpose you could use ``RemotePullTrigger`` trigger. To do so you will
|
||||
|
||||
During the next application run it will fetch repository from the specified url and will try to find packages there which can be used as local sources.
|
||||
|
||||
This feature can be also used to build packages which are not listed in AUR, the example of feature use can be found `here <https://github.com/arcan1s/ahriman/tree/master/recipes/pull>`__.
|
||||
This feature can be also used to build packages which are not listed in AUR, the example of the feature use can be found `here <https://github.com/arcan1s/ahriman/tree/master/recipes/pull>`__.
|
||||
|
||||
How to push updated PKGBUILDs to remote repository
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
@ -170,7 +170,7 @@ For that purpose you'd need to use another trigger called ``RemotePushTrigger``.
|
||||
[gitremote]
|
||||
push_url = https://github.com/username/repository
|
||||
|
||||
Unlike ``RemotePullTrigger`` trigger, the ``RemotePushTrigger`` more likely will require authorization. It is highly recommended to use application tokens for that instead of using your password (e.g. for Github you can generate tokens `here <https://github.com/settings/tokens>`__ with scope ``public_repo``). Authorization can be supplied by using authorization part of the url, e.g. ``https://key:token@github.com/username/repository``.
|
||||
Unlike ``RemotePullTrigger`` trigger, the ``RemotePushTrigger`` more likely will require authorization. It is highly recommended to use application tokens for that instead of using your password (e.g. for GitHub you can generate tokens `here <https://github.com/settings/tokens>`__ with scope ``public_repo``). Authorization can be supplied by using authorization part of the url, e.g. ``https://key:token@github.com/username/repository``.
|
||||
|
||||
How to change PKGBUILDs before build
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
@ -191,8 +191,10 @@ Alternatively you can create full-diff patches, which are calculated by using ``
|
||||
|
||||
#.
|
||||
Clone sources from AUR.
|
||||
|
||||
#.
|
||||
Make changes you would like to (e.g. edit ``PKGBUILD``, add external patches).
|
||||
|
||||
#.
|
||||
Run command
|
||||
|
||||
@ -213,11 +215,11 @@ The ``--variable`` argument accepts variables in shell like format: quotation an
|
||||
How to build package from official repository
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
So it is the same as adding any other package, but due to restrictions you must specify source explicitly, e.g.:
|
||||
It is the same as adding any other package, but due to restrictions you must specify source explicitly, e.g.:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
sudo -u ahriman ahriman package-add pacman -s repository
|
||||
sudo -u ahriman ahriman package-add pacman --source repository
|
||||
|
||||
This feature is heavily depends on local pacman cache. In order to use this feature it is recommended to either run ``pacman -Sy`` before the interaction or use internal application cache with ``--refresh`` flag.
|
||||
|
||||
@ -242,7 +244,7 @@ Normally the service handles VCS packages correctly, however it requires additio
|
||||
How to review changes before build
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
In this scenario, the update process must be separated to several stages. First, it is required to check updates:
|
||||
In this scenario, the update process must be separated into several stages. First, it is required to check updates:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
@ -274,7 +276,7 @@ Remove commands also remove any package files (patches, caches etc).
|
||||
How to sign repository
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Repository sign feature is available in several configurations. The recommended way is just to sign repository database file by single key instead of trying to sign each package. However, the steps are pretty same, just configuration is a bit differ. For more details about options kindly refer to :doc:`configuration reference <configuration>`.
|
||||
Repository sign feature is available in several configurations. The recommended way is just to sign repository database file by single key instead of trying to sign each package. However, the steps are pretty same, just configuration is a bit different. For more details about options kindly refer to :doc:`configuration reference <configuration>`.
|
||||
|
||||
#.
|
||||
First you would need to create the key on your local machine:
|
||||
@ -286,10 +288,10 @@ Repository sign feature is available in several configurations. The recommended
|
||||
This command will prompt you for several questions. Most of them may be left default, but you will need to fill real name and email address with some data. Because at the moment the service doesn't support passphrases, it must be left blank.
|
||||
|
||||
#.
|
||||
The command above will generate key and print its hash, something like ``8BE91E5A773FB48AC05CC1EDBED105AED6246B39``. Copy it.
|
||||
The command above will generate key and print its fingerprint, something like ``8BE91E5A773FB48AC05CC1EDBED105AED6246B39``. Copy it.
|
||||
|
||||
#.
|
||||
Export your private key by using the hash above:
|
||||
Export your private key by using the fingerprint above:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
@ -387,13 +389,15 @@ Example of the status page configuration is the following (status service is usi
|
||||
}
|
||||
}
|
||||
|
||||
Some more examples can be found in configuration `recipes <https://github.com/arcan1s/ahriman/tree/master/recipes>`__.
|
||||
|
||||
Docker image
|
||||
------------
|
||||
|
||||
We provide official images which can be found under:
|
||||
|
||||
* docker registry ``arcan1s/ahriman``;
|
||||
* ghcr.io registry ``ghcr.io/arcan1s/ahriman``;
|
||||
* ghcr.io registry ``ghcr.io/arcan1s/ahriman``.
|
||||
|
||||
These images are totally identical.
|
||||
|
||||
@ -417,7 +421,7 @@ The action can be specified during run, e.g.:
|
||||
|
||||
docker run --privileged -v /path/to/local/repo:/var/lib/ahriman arcan1s/ahriman:latest package-add ahriman --now
|
||||
|
||||
For more details please refer to docker FAQ.
|
||||
For more details please refer to the docker FAQ.
|
||||
|
||||
Environment variables
|
||||
^^^^^^^^^^^^^^^^^^^^^
|
||||
@ -440,7 +444,7 @@ The following environment variables are supported:
|
||||
* ``AHRIMAN_REPOSITORY_ROOT`` - repository root. Because of filesystem rights it is required to override default repository root. By default, it uses ``ahriman`` directory inside ahriman's home, which can be passed as mount volume.
|
||||
* ``AHRIMAN_UNIX_SOCKET`` - full path to unix socket which is used by web server, default is empty. Note that more likely you would like to put it inside ``AHRIMAN_REPOSITORY_ROOT`` directory (e.g. ``/var/lib/ahriman/ahriman/ahriman-web.sock``) or to ``/tmp``.
|
||||
* ``AHRIMAN_USER`` - ahriman user, usually must not be overwritten, default is ``ahriman``.
|
||||
* ``AHRIMAN_VALIDATE_CONFIGURATION`` - if set validate service configuration
|
||||
* ``AHRIMAN_VALIDATE_CONFIGURATION`` - if set (default) validate service configuration.
|
||||
|
||||
You can pass any of these variables by using ``-e`` argument, e.g.:
|
||||
|
||||
@ -457,7 +461,7 @@ There is special ``repo-daemon`` subcommand which emulates systemd timer and wil
|
||||
|
||||
docker run --privileged -v /path/to/local/repo:/var/lib/ahriman arcan1s/ahriman:latest repo-daemon
|
||||
|
||||
This command uses same rules as ``repo-update``, thus, e.g. requires ``--privileged`` flag.
|
||||
This command uses same rules as ``repo-update``, thus, e.g. requires ``--privileged`` flag. Chech also `examples <https://github.com/arcan1s/ahriman/tree/master/recipes/daemon>`__.
|
||||
|
||||
Web service setup
|
||||
^^^^^^^^^^^^^^^^^
|
||||
@ -484,6 +488,8 @@ Otherwise, you would need to pass ``AHRIMAN_PORT`` and mount container network t
|
||||
|
||||
docker run --privileged --net=host -e AHRIMAN_PORT=8080 -v /path/to/local/repo:/var/lib/ahriman arcan1s/ahriman:latest
|
||||
|
||||
Simple server with authentication can be found in `examples <https://github.com/arcan1s/ahriman/tree/master/recipes/web>`__ too.
|
||||
|
||||
Mutli-repository web service
|
||||
""""""""""""""""""""""""""""
|
||||
|
||||
@ -499,6 +505,8 @@ The command above will also create configuration for the repository named ``aur-
|
||||
|
||||
Note, however, that the command above is only required in case if the service is going to be used to run subprocesses. Otherwise, everything else (web interface, status, etc) will be handled as usual.
|
||||
|
||||
Configuration `example <https://github.com/arcan1s/ahriman/tree/master/recipes/multirepo>`__.
|
||||
|
||||
Non-x86_64 architecture setup
|
||||
-----------------------------
|
||||
|
||||
@ -614,8 +622,8 @@ There are several choices:
|
||||
|
||||
server {
|
||||
location / {
|
||||
root /var/lib/ahriman/repository/;
|
||||
autoindex on;
|
||||
root /var/lib/ahriman/repository/;
|
||||
}
|
||||
}
|
||||
|
||||
@ -630,7 +638,7 @@ There are several choices:
|
||||
[rsync]
|
||||
remote = 192.168.0.1:/srv/repo
|
||||
|
||||
After that just add ``/srv/repo`` to the ``pacman.conf`` as usual. You can also upload to S3 (``Server = https://s3.eu-central-1.amazonaws.com/repository/aur-clone/x86_64``) or to Github (``Server = https://github.com/ahriman/repository/releases/download/aur-clone-x86_64``).
|
||||
After that just add ``/srv/repo`` to the ``pacman.conf`` as usual. You can also upload to S3 (``Server = https://s3.eu-central-1.amazonaws.com/repository/aur-clone/x86_64``) or to GitHub (``Server = https://github.com/ahriman/repository/releases/download/aur-clone-x86_64``).
|
||||
|
||||
How to sync to S3
|
||||
^^^^^^^^^^^^^^^^^
|
||||
@ -643,10 +651,10 @@ How to sync to S3
|
||||
pacman -S python-boto3
|
||||
|
||||
#.
|
||||
Create a bucket.
|
||||
Create a bucket (e.g. ``repository``).
|
||||
|
||||
#.
|
||||
Create user with write access to the bucket:
|
||||
Create an user with write access to the bucket:
|
||||
|
||||
.. code-block::
|
||||
|
||||
@ -698,7 +706,7 @@ In order to configure S3 on custom domain with SSL (and some other features, lik
|
||||
|
||||
#. Configure S3 as described above.
|
||||
#. In bucket properties, enable static website hosting with hosting type "Host a static website".
|
||||
#. Go to AWS Certificate Manager and create public ceritificate on your domain. Validate domain as suggested.
|
||||
#. Go to AWS Certificate Manager and create public certificate on your domain. Validate domain as suggested.
|
||||
#. Go to CloudFront and create distribution. The following settings are required:
|
||||
|
||||
* Origin domain choose S3 bucket.
|
||||
@ -708,13 +716,15 @@ In order to configure S3 on custom domain with SSL (and some other features, lik
|
||||
|
||||
#. Point DNS record to CloudFront address.
|
||||
|
||||
How to sync to Github releases
|
||||
How to sync to GitHub releases
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
#.
|
||||
Create a repository.
|
||||
|
||||
#.
|
||||
`Create API key <https://github.com/settings/tokens>`__ with scope ``public_repo``.
|
||||
|
||||
#.
|
||||
Configure the service as following:
|
||||
|
||||
@ -794,7 +804,7 @@ How to post build report to telegram
|
||||
yay -S --asdeps python-jinja
|
||||
|
||||
#.
|
||||
Register bot in telegram. You can do it by talking with `@BotFather <https://t.me/botfather>`__. For more details please refer to `official documentation <https://core.telegram.org/bots>`__.
|
||||
Register bot in telegram. You can do it by starting chat with `@BotFather <https://t.me/botfather>`__. For more details please refer to `official documentation <https://core.telegram.org/bots>`__.
|
||||
|
||||
#.
|
||||
Optionally (if you want to post message in chat):
|
||||
@ -825,9 +835,9 @@ If you did everything fine you should receive the message with the next update.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
curl 'https://api.telegram.org/bot${CHAT_ID}/sendMessage?chat_id=${API_KEY}&text=hello'
|
||||
curl 'https://api.telegram.org/bot{api_key}/sendMessage?chat_id={chat_id}&text=hello'
|
||||
|
||||
(replace ``${CHAT_ID}`` and ``${API_KEY}`` with the values from configuration).
|
||||
(replace ``{chat_id}`` and ``{api_key}`` with the values from configuration).
|
||||
|
||||
Distributed builds
|
||||
------------------
|
||||
@ -879,6 +889,8 @@ Worker nodes configuration
|
||||
* Worker #1: ``A``.
|
||||
* Worker #2: ``B`` and ``C``.
|
||||
|
||||
Hint: ``repo-tree`` subcommand provides ``--partitions`` argument.
|
||||
|
||||
#.
|
||||
Each worker must be configured to upload files to master node:
|
||||
|
||||
@ -950,7 +962,7 @@ You can sign packages on worker nodes and then signatures will be synced to mast
|
||||
|
||||
Note, however, that in this case, signatures will not be validated on master node and just will be copied to repository tree.
|
||||
|
||||
If you would like to sign only database files (aka repository sign), it has to be configured on master node only as usual, e.g.:
|
||||
If you would like to sign only database files (aka repository sign), it has to be configured only on master node as usual, e.g.:
|
||||
|
||||
.. code-block:: ini
|
||||
|
||||
@ -966,7 +978,7 @@ Master node config (``master.ini``) as:
|
||||
.. code-block:: ini
|
||||
|
||||
[auth]
|
||||
target = mapping
|
||||
target = configuration
|
||||
|
||||
[web]
|
||||
enable_archive_upload = yes
|
||||
@ -1003,7 +1015,7 @@ The user ``worker-user`` has been created additionally. Worker node config (``wo
|
||||
[build]
|
||||
triggers = ahriman.core.gitremote.RemotePullTrigger ahriman.core.upload.UploadTrigger ahriman.core.report.ReportTrigger ahriman.core.gitremote.RemotePushTrigger
|
||||
|
||||
The address above (``http://172.17.0.1:8080``) is something available for worker container.
|
||||
The address above (``http://172.17.0.1:8080``) is somewhat available for worker container.
|
||||
|
||||
Command to run worker node:
|
||||
|
||||
@ -1070,7 +1082,7 @@ In this example, all instances are run on the same machine with address ``172.17
|
||||
.. code-block:: ini
|
||||
|
||||
[auth]
|
||||
target = mapping
|
||||
target = configuration
|
||||
|
||||
[status]
|
||||
username = builder-user
|
||||
@ -1094,7 +1106,7 @@ Worker nodes (applicable for all workers) config (``worker.ini``) as:
|
||||
.. code-block:: ini
|
||||
|
||||
[auth]
|
||||
target = mapping
|
||||
target = configuration
|
||||
|
||||
[status]
|
||||
address = http://172.17.0.1:8080
|
||||
@ -1126,21 +1138,21 @@ Command to run worker nodes (considering there will be two workers, one is on ``
|
||||
docker run --privileged -p 8081:8081 -e AHRIMAN_PORT=8081 -v worker.ini:/etc/ahriman.ini.d/overrides.ini arcan1s/ahriman:latest web
|
||||
docker run --privileged -p 8082:8082 -e AHRIMAN_PORT=8082 -v worker.ini:/etc/ahriman.ini.d/overrides.ini arcan1s/ahriman:latest web
|
||||
|
||||
Unlike the previous setup, it doesn't require to mount repository root for ``worker`` nodes, because ``worker`` nodes don't use it anyway.
|
||||
Unlike the previous setup, it doesn't require to mount repository root for ``worker`` nodes, because they don't use it anyway.
|
||||
|
||||
Check proof-of-concept setup `here <https://github.com/arcan1s/ahriman/tree/master/recipes/distributed>`__.
|
||||
|
||||
Addition of new package, package removal, repository update
|
||||
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
|
||||
|
||||
In all scenarios, update process must be run only on ``master`` node. Unlike the setup described above, automatic update must be enabled only for ``master`` node also.
|
||||
In all scenarios, update process must be run only on ``master`` node. Unlike the manually distributed packages described above, automatic update must be enabled only for ``master`` node.
|
||||
|
||||
Known limitations
|
||||
"""""""""""""""""
|
||||
|
||||
* Workers don't support local packages. However, it is possible to build custom packages by providing sources by using ``ahriman.core.gitremote.RemotePullTrigger`` trigger.
|
||||
* No dynamic nodes discovery. In case if one of worker nodes is unavailable, the build process will fail.
|
||||
* No pkgrel bump on conflicts. Well, it works, however, it isn't guaranteed.
|
||||
* No pkgrel bump on conflicts.
|
||||
* The identical user must be created for all workers. However, the ``master`` node user can be different from this one.
|
||||
|
||||
Maintenance packages
|
||||
@ -1252,7 +1264,7 @@ How to enable basic authorization
|
||||
yay -S --asdeps python-aiohttp-security python-aiohttp-session python-cryptography
|
||||
|
||||
#.
|
||||
Configure the service to enable authorization (``salt`` can be generated as any random string and optional):
|
||||
Configure the service to enable authorization:
|
||||
|
||||
.. code-block:: ini
|
||||
|
||||
@ -1260,17 +1272,17 @@ How to enable basic authorization
|
||||
target = configuration
|
||||
salt = somerandomstring
|
||||
|
||||
The ``salt`` parameter is optional, but recommended.
|
||||
The ``salt`` parameter is optional, but recommended, and can be set to any (random) string.
|
||||
|
||||
#.
|
||||
In order to provide access for reporting from application instances you can (recommended way) use unix sockets by configuring the following (note, that it requires ``python-requests-unixsocket`` package to be installed):
|
||||
In order to provide access for reporting from application instances you can (the recommended way) use unix sockets by the following configuration (note, that it requires ``python-requests-unixsocket`` package to be installed):
|
||||
|
||||
.. code-block:: ini
|
||||
|
||||
[web]
|
||||
unix_socket = /var/lib/ahriman/ahriman-web.sock
|
||||
|
||||
This socket path must be available for web service instance and must be available for application instances (e.g. in case if you are using docker container, see above, you need to be sure that the socket is passed to the root filesystem).
|
||||
This socket path must be available for web service instance and must be available for all application instances (e.g. in case if you are using docker container - see above - you need to make sure that the socket is passed to the root filesystem).
|
||||
|
||||
By the way, unix socket variable will be automatically set in case if ``--web-unix-socket`` argument is supplied to the ``setup`` subcommand.
|
||||
|
||||
@ -1303,6 +1315,7 @@ How to enable OAuth authorization
|
||||
|
||||
#.
|
||||
Create OAuth web application, download its ``client_id`` and ``client_secret``.
|
||||
|
||||
#.
|
||||
Guess what? Install dependencies:
|
||||
|
||||
@ -1388,7 +1401,7 @@ The service provides several commands aim to do easy repository backup and resto
|
||||
Use cases
|
||||
---------
|
||||
|
||||
There is collection of some specific recipes which can be found in `the repository <https://github.com/arcan1s/ahriman/tree/master/recipes>`__.
|
||||
There is a collection of some specific recipes which can be found in `the repository <https://github.com/arcan1s/ahriman/tree/master/recipes>`__.
|
||||
|
||||
Most of them can be run (``AHRIMAN_PASSWORD`` environment variable is required in the most setups) as simple as:
|
||||
|
||||
@ -1409,7 +1422,7 @@ Short answer - I do not know. Also for some references credits to `Alad <https:/
|
||||
`arch-repo-manager <https://github.com/Martchus/arch-repo-manager>`__
|
||||
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
|
||||
|
||||
Looks actually pretty good, in case if I would find it, I would probably didn't start this project, most of features (like web interface or additional helpers) are already implemented or planned to be. However, this project seems to be at early alpha stage (as for Nov 2022), written in C++ (not pro or con) and misses code documentation.
|
||||
Looks actually pretty good, in case if I would find it, I would probably didn't start this project; the most of features (like web interface or additional helpers) are already implemented or planned to be. However, this project seems to be at early alpha stage (as for Nov 2022), written in C++ (not pro or con) and misses documentation.
|
||||
|
||||
`archrepo2 <https://github.com/lilydjwg/archrepo2>`__
|
||||
"""""""""""""""""""""""""""""""""""""""""""""""""""""
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
Triggers
|
||||
========
|
||||
|
||||
The package provides ability to write custom extensions which will be run on (the most) actions, e.g. after updates. By default ahriman provides three types of extensions - reporting, files uploading and PKGBUILD syncronization. Each extension must derive from the ``ahriman.core.triggers.Trigger`` class and should implement at least one of the abstract methods:
|
||||
The package provides ability to write custom extensions which will be run on (the most) actions, e.g. after updates. By default ahriman provides three types of extensions - reporting, files uploading and PKGBUILD synchronization. Each extension must derive from the ``ahriman.core.triggers.Trigger`` class and should implement at least one of the abstract methods:
|
||||
|
||||
* ``on_result`` - trigger action which will be called after build process, the build result and the list of repository packages will be supplied as arguments.
|
||||
* ``on_start`` - trigger action which will be called right before the start of the application process.
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
[auth]
|
||||
target = mapping
|
||||
target = configuration
|
||||
|
||||
[web]
|
||||
enable_archive_upload = yes
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
[auth]
|
||||
target = mapping
|
||||
target = configuration
|
||||
|
||||
[build]
|
||||
workers = http://worker:8080 http://worker:8080
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
[auth]
|
||||
target = mapping
|
||||
target = configuration
|
||||
|
||||
[build]
|
||||
triggers = ahriman.core.upload.UploadTrigger ahriman.core.report.ReportTrigger
|
||||
|
||||
@ -1,2 +1,2 @@
|
||||
[auth]
|
||||
target = mapping
|
||||
target = configuration
|
||||
|
||||
@ -1,2 +1,2 @@
|
||||
[auth]
|
||||
target = mapping
|
||||
target = configuration
|
||||
|
||||
@ -1,2 +1,2 @@
|
||||
[auth]
|
||||
target = mapping
|
||||
target = configuration
|
||||
|
||||
@ -109,7 +109,8 @@ class Application(ApplicationPackages, ApplicationRepository):
|
||||
|
||||
def with_dependencies(self, packages: list[Package], *, process_dependencies: bool) -> list[Package]:
|
||||
"""
|
||||
add missing dependencies to list of packages
|
||||
add missing dependencies to list of packages. This will extract known packages, check dependencies of
|
||||
the supplied packages and add packages which are not presented in the list of known packages.
|
||||
|
||||
Args:
|
||||
packages(list[Package]): list of source packages of which dependencies have to be processed
|
||||
@ -118,6 +119,14 @@ class Application(ApplicationPackages, ApplicationRepository):
|
||||
Returns:
|
||||
list[Package]: updated packages list. Packager for dependencies will be copied from
|
||||
original package
|
||||
|
||||
Examples:
|
||||
In the most cases, in order to avoid build failure, it is required to add missing packages, which can be
|
||||
done by calling::
|
||||
|
||||
>>> application = ...
|
||||
>>> packages = application.with_dependencies(packages, process_dependencies=True)
|
||||
>>> application.print_updates(packages, log_fn=print)
|
||||
"""
|
||||
def missing_dependencies(source: Iterable[Package]) -> dict[str, str | None]:
|
||||
# append list of known packages with packages which are in current sources
|
||||
|
||||
@ -51,7 +51,7 @@ class ApplicationRepository(ApplicationProperties):
|
||||
|
||||
def clean(self, *, cache: bool, chroot: bool, manual: bool, packages: bool, pacman: bool) -> None:
|
||||
"""
|
||||
run all clean methods. Warning: some functions might not be available under non-root
|
||||
run all clean methods. Warning: some functions might not be available for non-root user
|
||||
|
||||
Args:
|
||||
cache(bool): clear directory with package caches
|
||||
@ -142,7 +142,11 @@ class ApplicationRepository(ApplicationProperties):
|
||||
def update(self, updates: Iterable[Package], packagers: Packagers | None = None, *,
|
||||
bump_pkgrel: bool = False) -> Result:
|
||||
"""
|
||||
run package updates
|
||||
run package updates. This method will separate update in the several steps:
|
||||
|
||||
#. Check already built packages.
|
||||
#. Construct builder instance.
|
||||
#. Delegate build process to the builder instance (either remote or local).
|
||||
|
||||
Args:
|
||||
updates(Iterable[Package]): list of packages to update
|
||||
|
||||
@ -126,7 +126,7 @@ class RemoteUpdater(Updater):
|
||||
Result: update result
|
||||
"""
|
||||
payload = {
|
||||
"increment": bump_pkgrel,
|
||||
"increment": False, # force disable increment because it doesn't work yet
|
||||
"packager": packagers.default if packagers is not None else None,
|
||||
"packages": [package.base for package in updates],
|
||||
"patches": [], # might be used later
|
||||
|
||||
@ -135,12 +135,12 @@ class Lock(LazyLogging):
|
||||
"""
|
||||
default workflow is the following:
|
||||
|
||||
1. Check user UID
|
||||
2. Check if there is lock file
|
||||
3. Check web status watcher status
|
||||
4. Wait for lock file to be free
|
||||
5. Create lock file and directory tree
|
||||
6. Report to status page if enabled
|
||||
#. Check user UID
|
||||
#. Check if there is lock file
|
||||
#. Check web status watcher status
|
||||
#. Wait for lock file to be free
|
||||
#. Create lock file and directory tree
|
||||
#. Report to status page if enabled
|
||||
|
||||
Returns:
|
||||
Self: always instance of self
|
||||
|
||||
@ -32,7 +32,7 @@ class Remote(SyncHttpClient):
|
||||
|
||||
>>> from ahriman.core.alpm.remote import AUR, Official
|
||||
>>>
|
||||
>>> package = AUR.info("ahriman", pacman=pacman)
|
||||
>>> package = AUR.info("ahriman")
|
||||
>>> search_result = Official.multisearch("pacman", "manager", pacman=pacman)
|
||||
|
||||
Differnece between :func:`search()` and :func:`multisearch()` is that :func:`search()` passes all arguments to
|
||||
|
||||
@ -76,7 +76,7 @@ class Repo(LazyLogging):
|
||||
|
||||
def init(self) -> None:
|
||||
"""
|
||||
create empty repository database
|
||||
create empty repository database. It just calls add with empty arguments
|
||||
"""
|
||||
check_output("repo-add", *self.sign_args, str(self.repo_path),
|
||||
cwd=self.paths.repository, logger=self.logger, user=self.uid)
|
||||
|
||||
@ -32,7 +32,7 @@ from ahriman.models.repository_paths import RepositoryPaths
|
||||
|
||||
class Sources(LazyLogging):
|
||||
"""
|
||||
helper to download package sources (PKGBUILD etc...)
|
||||
helper to download package sources (PKGBUILD etc...) and perform some operations with git
|
||||
|
||||
Attributes:
|
||||
DEFAULT_BRANCH(str): (class attribute) default branch to process git repositories.
|
||||
|
||||
@ -75,10 +75,9 @@ def test_update(remote_updater: RemoteUpdater, package_ahriman: Package, mocker:
|
||||
request_mock.assert_called_once_with("POST", remote_updater._update_url(worker),
|
||||
params=remote_updater.repository_id.query(),
|
||||
json={
|
||||
"increment": True,
|
||||
"increment": False,
|
||||
"packager": "username",
|
||||
"packages": [package_ahriman.base],
|
||||
"patches": [],
|
||||
"refresh": True,
|
||||
}
|
||||
)
|
||||
})
|
||||
|
||||
Loading…
Reference in New Issue
Block a user