arcanis/ahriman
1
0
Fork 0
mirror of https://github.com/arcan1s/ahriman.git synced 2025-04-24 15:27:17 +00:00
Code Issues Projects Releases Wiki Activity

Add gitremote triggers (#68)

Browse Source 
* add gitremote pull trigger

* add push gitremote trigger

* docs update
This commit is contained in:
Evgenii Alekseev 2022-10-18 01:46:27 +03:00 committed by GitHub
parent fc0d8387df
commit 342b3cb652
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
60 changed files with 722 additions and 130 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.pylintrc
View File

@ -82,7 +82,8 @@ disable=raw-checker-failed,
fixme,
too-many-arguments,
duplicate-code,
cyclic-import
cyclic-import,
confusing-with-statement,
# Enable the message, report, category or checker with the given id(s). You can

2
Makefile
View File

@ -56,4 +56,4 @@ version:
ifndef VERSION
$(error VERSION is required, but not set)
endif
sed -i '/__version__ = .*/s/[^"][^)]*/__version__ = "$(VERSION)"/' src/ahriman/version.py
sed -i 's/^__version__ = .*/__version__ = "$(VERSION)"/' src/ahriman/version.py

2
docs/ahriman.1
View File

@ -391,7 +391,7 @@ just perform check for packages without rebuild process itself
.TP
\fB\-\-from\-database\fR
read packages from database instead of filesystem. This feature in particular is required in case if you would like to
restore repository from another repository instance. Note however that in order to restore packages you need to have
restore repository from another repository instance. Note, however, that in order to restore packages you need to have
original ahriman instance run with web service and have run repo\-update at least once.
.TP

29
docs/ahriman.core.gitremote.rst Normal file
View File

@ -0,0 +1,29 @@
ahriman.core.gitremote package
==============================
Submodules
----------
ahriman.core.gitremote.remote\_pull\_trigger module
---------------------------------------------------
.. automodule:: ahriman.core.gitremote.remote_pull_trigger
:members:
:no-undoc-members:
:show-inheritance:
ahriman.core.gitremote.remote\_push\_trigger module
---------------------------------------------------
.. automodule:: ahriman.core.gitremote.remote_push_trigger
:members:
:no-undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: ahriman.core.gitremote
:members:
:no-undoc-members:
:show-inheritance:

1
docs/ahriman.core.rst
View File

@ -12,6 +12,7 @@ Subpackages
ahriman.core.build_tools
ahriman.core.database
ahriman.core.formatters
ahriman.core.gitremote
ahriman.core.report
ahriman.core.repository
ahriman.core.sign

9
docs/architecture.rst
View File

@ -35,12 +35,13 @@ This package contains everything which is required for any time of application r
* ``ahriman.core.build_tools`` is a package which provides wrapper for ``devtools`` commands.
* ``ahriman.core.database`` is everything including data and schema migrations for database.
* ``ahriman.core.formatters`` package provides ``Printer`` sub-classes for printing data (e.g. package properties) to stdout which are used by some handlers.
* ``ahriman.core.report`` is a package with reporting classes. Usually it must be called by ``ahriman.core.report.Report.load`` method.
* ``ahriman.core.gitremote`` is a package with remote PKGBUILD triggers. Should not be called directly.
* ``ahriman.core.report`` is a package with reporting triggers. Should not be called directly.
* ``ahriman.core.repository`` contains several traits and base repository (``ahriman.core.repository.Repository`` class) implementation.
* ``ahriman.core.sign`` package provides sign feature (only gpg calls are available).
* ``ahriman.core.status`` contains helpers and watcher class which are required for web application. Reporter must be initialized by using ``ahriman.core.status.client.Client.load`` method.
* ``ahriman.core.triggers`` package contains base trigger classes. Classes from this package must be imported in order to implement user extensions. In fact, ``ahriman.core.report`` and ``ahriman.core.upload`` use this package.
* ``ahriman.core.upload`` package provides sync feature, must be called by ``ahriman.core.upload.Upload.load`` method.
* ``ahriman.core.upload`` package provides sync feature, should not be called directly.
This package also provides some generic functions and classes which may be used by other packages:
@ -202,12 +203,14 @@ In order to configure users there are special commands.
Triggers
^^^^^^^^
Triggers are extensions which can be used in order to perform any actions after the update process. The package provides two default extensions - one is report generation and another one is remote upload feature.
Triggers are extensions which can be used in order to perform any actions on application start, after the update process and, finally, before the application exit. The package provides two default extensions - one is report generation and another one is remote upload feature.
The main idea is to load classes by their full path (e.g. ``ahriman.core.upload.UploadTrigger``) by using ``importlib``: get the last part of the import and treat it as class name, join remain part by ``.`` and interpret as module path, import module and extract attribute from it.
The loaded triggers will be called with ``ahriman.models.result.Result`` and ``List[Packages]`` arguments, which describes the process result and current repository packages respectively. Any exception raised will be suppressed and will generate an exception message in logs.
In addition triggers can implement ``on_start`` and ``on_stop`` actions which will be called on the application start and right before the application exit. The ``on_start`` action is usually being called from handlers directly in order to make sure that no trigger will be run when it is not required (e.g. on user management). As soon as ``on_start`` action is called, the additional flag will be set; ``ahriman.core.triggers.TriggerLoader`` class implements ``__del__`` method in which, if the flag is set, the ``on_stop`` actions will be called.
For more details how to deal with the triggers, refer to :doc:`documentation <triggers>` and modules descriptions.
Remote synchronization

12
docs/configuration.rst
View File

@ -75,6 +75,18 @@ Settings for signing packages or repository. Group name can refer to architectur
* ``key`` - default PGP key, string, required. This key will also be used for database signing if enabled.
* ``key_*`` settings - PGP key which will be used for specific packages, string, optional. For example, if there is ``key_yay`` option the specified key will be used for yay package and default key for others.
``gitremote`` group
-------------------
Remote git source synchronization settings. Unlike ``Upload`` triggers those triggers are used for PKGBUILD synchronization - e.g. fetch from remote repository PKGBUILDs before updating process or pulling updated PKGBUILDs to the remote repository.
Both urls support authorization; to do so you'd need to prefix the url with authorization part, e.g. ``https://key:token@github.com/arcan1s/ahriman.git``. It is highly recommended to use application tokens instead of your user authorization details.
* ``pull_url`` - url of the remote repository from which PKGBUILDs can be pulled before build process, string, required.
* ``pull_branch`` - branch of the remote repository from which PKGBUILDs can be pulled before build process, string, optional, default is ``master``.
* ``push_url`` - url of the remote repository to which PKGBUILDs should be pushed after build process, string, required.
* ``push_branch`` - branch of the remote repository to which PKGBUILDs should be pushed after build process, string, optional, default is ``master``.
``report`` group
----------------

47
docs/faq.rst
View File

@ -112,6 +112,53 @@ TL;DR
Before using this command you will need to create local directory, put ``PKGBUILD`` there and generate ``.SRCINFO`` by using ``makepkg --printsrcinfo > .SRCINFO`` command. These packages will be stored locally and *will be ignored* during automatic update; in order to update the package you will need to run ``package-add`` command again.
Err, I have remote repository with PKGBUILDs and would like to get versions from there automatically
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
For that purpose you could use ``RemotePullTrigger`` trigger. To do so you will need:
#.
Append ``triggers`` option in ``build`` section with the following line:
.. code-block:: ini
[build]
triggers = ahriman.core.gitremote.RemotePullTrigger
#.
Configure trigger as following:
.. code-block:: ini
[gitremote]
pull_url = https://github.com/username/repository
During the next application run it will fetch repository from the specified url and will try to find packages there which can be used as local sources.
I would like to push PKGBUILDs to the remote repository
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
For that purpose you'd need to use another trigger called ``RemotePushTrigger``. Configure it as following:
#.
Append ``triggers`` option in ``build`` section with the trigger name:
.. code-block:: ini
[build]
triggers = ahriman.core.gitremote.RemotePushTrigger
#.
Configure trigger as following:
.. code-block:: ini
[gitremote]
push_url = https://github.com/username/repository
Unlike ``RemotePullTrigger`` trigger, the ``RemotePushTrigger`` more likely will require authorization. It is highly recommended to use application tokens for that instead of using your password (e.g. for Github you can generate tokens `here <https://github.com/settings/tokens>`_ with scope ``public_repo``). Authorization can be supplied by using authorization part of the url, e.g. ``https://key:token@github.com/username/repository``.
But I just wanted to change PKGBUILD from AUR a bit!
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

43
docs/triggers.rst
View File

@ -1,12 +1,45 @@
Triggers
========
The package provides ability to write custom extensions which will be run on (the most) actions, e.g. after updates. By default ahriman provides two types of extensions - reporting and files uploading. Each extension must derive from the ``ahriman.core.triggers.Trigger`` class and implement ``run`` method
The package provides ability to write custom extensions which will be run on (the most) actions, e.g. after updates. By default ahriman provides three types of extensions - reporting, files uploading and PKGBUILD syncronization. Each extension must derive from the ``ahriman.core.triggers.Trigger`` class and should implement at least one of the abstract methods:
* ``on_result`` - trigger action which will be called after build process, the build result and the list of repository packages will be supplied as arguments.
* ``on_start`` - trigger action which will be called right before the start of the application process.
* ``on_stop`` - action which will be called right before the exit.
Note, it isn't required to implement all of those methods (or even one of them), however, it is highly recommended to avoid trigger actions in ``__init__`` method as it will be run on any application start (e.g. even if you are just searching in AUR).
Built-in triggers
-----------------
For the configuration details and settings explanation kindly refer to the :doc:`documentation <configuration>`.
``ahriman.core.gitremote.RemotePullTrigger``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This trigger will be called before any action (``on_start``) and pulls remote PKGBUILD repository locally; after that it copies found PKGBUILDs from the cloned repository to the local cache. It is useful in case if you have patched PGKBUILDs (or even missing in AUR) which you would like to use for package building and, technically, just simplifies the local package building.
In order to update those packages you would need to clone your repository separately, make changes in PKGBUILD (e.g. bump version and update checksums), commit them and push back. On the next ahriman's repository update, it will pull changes you commited and will perform package update.
``ahriman.core.gitremote.RemotePushTrigger``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This trigger will be called right after build process (``on_result``). It will pick PKGBUILDs for the updated packages, pull them (together with any other files) and commit and push changes to remote repository. No real use cases, but the most of user repositories do it.
``ahriman.core.report.ReportTrigger``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Trigger which can be used for reporting. It implements ``on_result`` method and thus being called on each build update and generates report (e.g. html, telegram etc) according to the current settings.
``ahriman.core.upload.UploadTrigger``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This trigger takes build result (``on_result``) and performs syncing of the local packages to the remote mirror (e.g. S3 or just by rsync).
Trigger example
---------------
Lets consider example of reporting trigger (e.g. `slack <https://slack.com/>`_, which provides easy HTTP API for integration triggers).
Lets consider example of reporting trigger (e.g. `slack <https://slack.com/>`_, which provides easy HTTP API for integration triggers).gre
In order to post message to slack we will need a specific trigger url (something like ``https://hooks.slack.com/services/company_id/trigger_id``), channel (e.g. ``#archrepo``) and username (``repo-bot``).
@ -49,12 +82,12 @@ Obviously you can implement the specified method in class, but for guide purpose
self.channel = configuration.get("slack", "channel")
self.username = configuration.get("slack", "username")
def run(self, result, packages):
def on_result(self, result, packages):
notify(result, self.slack_url, self.channel, self.username)
Setup the trigger
-----------------
^^^^^^^^^^^^^^^^^
First, put the trigger in any path it can be exported, e.g. by packing the resource into python package (which will lead to import path as ``package.slack_reporter.SlackReporter``) or just put file somewhere it can be accessed by application (e.g. ``/usr/local/lib/slack_reporter.py.SlackReporter``.
First, put the trigger in any path it can be exported, e.g. by packing the resource into python package (which will lead to import path as ``package.slack_reporter.SlackReporter``) or just put file somewhere it can be accessed by application (e.g. ``/usr/local/lib/slack_reporter.py.SlackReporter``).
After that run application as usual and receive notification in your slack channel.

32
src/ahriman/application/application/application.py
View File

@ -49,15 +49,6 @@ class Application(ApplicationPackages, ApplicationRepository):
be used instead.
"""
def _finalize(self, result: Result) -> None:
"""
generate report and sync to remote server
Args:
result(Result): build result
"""
self.repository.process_triggers(result)
def _known_packages(self) -> Set[str]:
"""
load packages from repository and pacman repositories
@ -73,3 +64,26 @@ class Application(ApplicationPackages, ApplicationRepository):
known_packages.update(properties.provides)
known_packages.update(self.repository.pacman.all_packages())
return known_packages
def on_result(self, result: Result) -> None:
"""
generate report and sync to remote server
Args:
result(Result): build result
"""
packages = self.repository.packages()
self.repository.triggers.on_result(result, packages)
def on_start(self) -> None:
"""
run triggers on start of the application
"""
self.repository.triggers.on_start()
def on_stop(self) -> None:
"""
run triggers on stop of the application. Note, however, that in most cases this method should not be called
directly as it will be called after on_start action
"""
self.repository.triggers.on_stop()

29
src/ahriman/application/application/application_packages.py
View File

@ -37,18 +37,6 @@ class ApplicationPackages(ApplicationProperties):
package control class
"""
def _finalize(self, result: Result) -> None:
"""
generate report and sync to remote server
Args:
result(Result): build result
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def _known_packages(self) -> Set[str]:
"""
load packages from repository and pacman repositories
@ -61,6 +49,18 @@ class ApplicationPackages(ApplicationProperties):
"""
raise NotImplementedError
def on_result(self, result: Result) -> None:
"""
generate report and sync to remote server
Args:
result(Result): build result
Raises:
NotImplementedError: not implemented method
"""
raise NotImplementedError
def _add_archive(self, source: str, *_: Any) -> None:
"""
add package from archive
@ -86,8 +86,7 @@ class ApplicationPackages(ApplicationProperties):
self.database.build_queue_insert(package)
self.database.remote_update(package)
with TemporaryDirectory(ignore_cleanup_errors=True) as dir_name, \
(local_dir := Path(dir_name)): # pylint: disable=confusing-with-statement
with TemporaryDirectory(ignore_cleanup_errors=True) as dir_name, (local_dir := Path(dir_name)):
Sources.load(local_dir, package, self.database.patches_get(package.base), self.repository.paths)
self._process_dependencies(local_dir, known_packages, without_dependencies)
@ -187,4 +186,4 @@ class ApplicationPackages(ApplicationProperties):
names(Iterable[str]): list of packages (either base or name) to remove
"""
self.repository.process_remove(names)
self._finalize(Result())
self.on_result(Result())

6
src/ahriman/application/application/application_repository.py
View File

@ -35,7 +35,7 @@ class ApplicationRepository(ApplicationProperties):
repository control class
"""
def _finalize(self, result: Result) -> None:
def on_result(self, result: Result) -> None:
"""
generate report and sync to remote server
@ -89,7 +89,7 @@ class ApplicationRepository(ApplicationProperties):
self.update([])
# sign repository database if set
self.repository.sign.process_sign_repository(self.repository.repo.repo_path)
self._finalize(Result())
self.on_result(Result())
def unknown(self) -> List[str]:
"""
@ -139,7 +139,7 @@ class ApplicationRepository(ApplicationProperties):
if not paths:
return # don't need to process if no update supplied
update_result = self.repository.process_update(paths)
self._finalize(result.merge(update_result))
self.on_result(result.merge(update_result))
# process built packages
build_result = Result()

1
src/ahriman/application/handlers/add.py
View File

@ -45,6 +45,7 @@ class Add(Handler):
unsafe(bool): if set no user check will be performed before path creation
"""
application = Application(architecture, configuration, no_report, unsafe)
application.on_start()
application.add(args.package, args.source, args.without_dependencies)
if not args.now:
return

5
src/ahriman/application/handlers/clean.py
View File

@ -44,5 +44,6 @@ class Clean(Handler):
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
Application(architecture, configuration, no_report, unsafe).clean(
args.cache, args.chroot, args.manual, args.packages)
application = Application(architecture, configuration, no_report, unsafe)
application.on_start()
application.clean(args.cache, args.chroot, args.manual, args.packages)

4
src/ahriman/application/handlers/key_import.py
View File

@ -46,5 +46,5 @@ class KeyImport(Handler):
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
Application(architecture, configuration, no_report, unsafe).repository.sign.key_import(
args.key_server, args.key)
application = Application(architecture, configuration, no_report, unsafe)
application.repository.sign.key_import(args.key_server, args.key)

1
src/ahriman/application/handlers/patch.py
View File

@ -50,6 +50,7 @@ class Patch(Handler):
unsafe(bool): if set no user check will be performed before path creation
"""
application = Application(architecture, configuration, no_report, unsafe)
application.on_start()
if args.action == Action.List:
Patch.patch_set_list(application, args.package, args.exit_code)

2
src/ahriman/application/handlers/rebuild.py
View File

@ -49,6 +49,8 @@ class Rebuild(Handler):
depends_on = set(args.depends_on) if args.depends_on else None
application = Application(architecture, configuration, no_report, unsafe)
application.on_start()
if args.from_database:
updates = Rebuild.extract_packages(application)
else:

4
src/ahriman/application/handlers/remove.py
View File

@ -44,4 +44,6 @@ class Remove(Handler):
no_report(bool): force disable reporting
unsafe(bool): if set no user check will be performed before path creation
"""
Application(architecture, configuration, no_report, unsafe).remove(args.package)
application = Application(architecture, configuration, no_report, unsafe)
application.on_start()
application.remove(args.package)

1
src/ahriman/application/handlers/remove_unknown.py
View File

@ -46,6 +46,7 @@ class RemoveUnknown(Handler):
unsafe(bool): if set no user check will be performed before path creation
"""
application = Application(architecture, configuration, no_report, unsafe)
application.on_start()
unknown_packages = application.unknown()
if args.dry_run:

3
src/ahriman/application/handlers/triggers.py
View File

@ -49,4 +49,5 @@ class Triggers(Handler):
if args.trigger:
loader = application.repository.triggers
loader.triggers = [loader.load_trigger(trigger) for trigger in args.trigger]
application.repository.process_triggers(Result())
application.on_start()
application.on_result(Result())

1
src/ahriman/application/handlers/update.py
View File

@ -45,6 +45,7 @@ class Update(Handler):
unsafe(bool): if set no user check will be performed before path creation
"""
application = Application(architecture, configuration, no_report, unsafe)
application.on_start()
packages = application.updates(args.package, args.no_aur, args.no_local, args.no_manual, args.no_vcs,
Update.log_fn(application, args.dry_run))
Update.check_if_empty(args.exit_code, not packages)

31
src/ahriman/core/build_tools/sources.py
View File

@ -17,6 +17,7 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
import datetime
import shutil
from pathlib import Path
@ -170,6 +171,22 @@ class Sources(LazyLogging):
diff = instance.diff(sources_dir)
return f"{diff}\n" # otherwise, patch will be broken
@staticmethod
def push(sources_dir: Path, remote: RemoteSource, *pattern: str) -> None:
"""
commit selected changes and push files to the remote repository
Args:
sources_dir(Path): local path to git repository
remote(RemoteSource): remote target, branch and url
*pattern(str): glob patterns
"""
instance = Sources()
instance.add(sources_dir, *pattern)
instance.commit(sources_dir)
Sources._check_output("git", "push", remote.git_url, remote.branch,
exception=None, cwd=sources_dir, logger=instance.logger)
def add(self, sources_dir: Path, *pattern: str) -> None:
"""
track found files via git
@ -190,6 +207,20 @@ class Sources(LazyLogging):
*[str(fn.relative_to(sources_dir)) for fn in found_files],
exception=None, cwd=sources_dir, logger=self.logger)
def commit(self, sources_dir: Path, commit_message: Optional[str] = None) -> None:
"""
commit changes
Args:
sources_dir(Path): local path to git repository
commit_message(Optional[str]): optional commit message if any. If none set, message will be generated
according to the current timestamp
"""
if commit_message is None:
commit_message = f"Autogenerated commit at {datetime.datetime.utcnow()}"
Sources._check_output("git", "commit", "--all", "--message", commit_message,
exception=None, cwd=sources_dir, logger=self.logger)
def diff(self, sources_dir: Path) -> str:
"""
generate diff from the current version and write it to the output file

21
src/ahriman/core/gitremote/__init__.py Normal file
View File

@ -0,0 +1,21 @@
#
# copyright (c) 2021-2022 ahriman team.
#
# this file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# this program is free software: you can redistribute it and/or modify
# it under the terms of the gnu general public license as published by
# the free software foundation, either version 3 of the license, or
# (at your option) any later version.
#
# this program is distributed in the hope that it will be useful,
# but without any warranty; without even the implied warranty of
# merchantability or fitness for a particular purpose. see the
# gnu general public license for more details.
#
# you should have received a copy of the gnu general public license
# along with this program. if not, see <http://www.gnu.org/licenses/>.
#
from ahriman.core.gitremote.remote_pull_trigger import RemotePullTrigger
from ahriman.core.gitremote.remote_push_trigger import RemotePushTrigger

86
src/ahriman/core/gitremote/remote_pull_trigger.py Normal file
View File

@ -0,0 +1,86 @@
#
# copyright (c) 2021-2022 ahriman team.
#
# this file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# this program is free software: you can redistribute it and/or modify
# it under the terms of the gnu general public license as published by
# the free software foundation, either version 3 of the license, or
# (at your option) any later version.
#
# this program is distributed in the hope that it will be useful,
# but without any warranty; without even the implied warranty of
# merchantability or fitness for a particular purpose. see the
# gnu general public license for more details.
#
# you should have received a copy of the gnu general public license
# along with this program. if not, see <http://www.gnu.org/licenses/>.
#
import shutil
from pathlib import Path
from tempfile import TemporaryDirectory
from ahriman.core.build_tools.sources import Sources
from ahriman.core.configuration import Configuration
from ahriman.core.triggers import Trigger
from ahriman.core.util import walk
from ahriman.models.package_source import PackageSource
from ahriman.models.remote_source import RemoteSource
class RemotePullTrigger(Trigger):
"""
trigger for fetching PKGBUILDs from remote repository
Attributes:
remote_source(RemoteSource): repository remote source (remote pull url and branch)
repository_paths(RepositoryPaths): repository paths instance
"""
def __init__(self, architecture: str, configuration: Configuration) -> None:
"""
default constructor
Args:
architecture(str): repository architecture
configuration(Configuration): configuration instance
"""
Trigger.__init__(self, architecture, configuration)
self.remote_source = RemoteSource(
git_url=configuration.get("gitremote", "pull_url"),
web_url="",
path=".",
branch=configuration.get("gitremote", "pull_branch", fallback="master"),
source=PackageSource.Local,
)
self.repository_paths = configuration.repository_paths
def on_start(self) -> None:
"""
trigger action which will be called at the start of the application
"""
self.repo_clone()
def repo_clone(self) -> None:
"""
clone repository from remote source
"""
with TemporaryDirectory(ignore_cleanup_errors=True) as dir_name, (clone_dir := Path(dir_name)):
Sources.fetch(clone_dir, self.remote_source)
self.repo_copy(clone_dir)
def repo_copy(self, clone_dir: Path) -> None:
"""
copy directories from cloned remote source to local cache
Args:
clone_dir(Path): path to temporary cloned directory
"""
for pkgbuild_path in filter(lambda path: path.name == "PKGBUILD", walk(clone_dir)):
cloned_pkgbuild_dir = pkgbuild_path.parent
package_base = cloned_pkgbuild_dir.name
local_pkgbuild_dir = self.repository_paths.cache_for(package_base)
shutil.copytree(cloned_pkgbuild_dir, local_pkgbuild_dir, dirs_exist_ok=True)
Sources.init(local_pkgbuild_dir) # initialized git repository is required for local sources

110
src/ahriman/core/gitremote/remote_push_trigger.py Normal file
View File

@ -0,0 +1,110 @@
#
# copyright (c) 2021-2022 ahriman team.
#
# this file is part of ahriman
# (see https://github.com/arcan1s/ahriman).
#
# this program is free software: you can redistribute it and/or modify
# it under the terms of the gnu general public license as published by
# the free software foundation, either version 3 of the license, or
# (at your option) any later version.
#
# this program is distributed in the hope that it will be useful,
# but without any warranty; without even the implied warranty of
# merchantability or fitness for a particular purpose. see the
# gnu general public license for more details.
#
# you should have received a copy of the gnu general public license
# along with this program. if not, see <http://www.gnu.org/licenses/>.
#
import shutil
from pathlib import Path
from tempfile import TemporaryDirectory
from typing import Generator, Iterable
from ahriman.core.build_tools.sources import Sources
from ahriman.core.configuration import Configuration
from ahriman.core.triggers import Trigger
from ahriman.models.package import Package
from ahriman.models.package_source import PackageSource
from ahriman.models.remote_source import RemoteSource
from ahriman.models.result import Result
class RemotePushTrigger(Trigger):
"""
trigger for syncing PKGBUILDs to remote repository
Attributes:
remote_source(RemoteSource): repository remote source (remote pull url and branch)
"""
def __init__(self, architecture: str, configuration: Configuration) -> None:
"""
default constructor
Args:
architecture(str): repository architecture
configuration(Configuration): configuration instance
"""
Trigger.__init__(self, architecture, configuration)
self.remote_source = RemoteSource(
git_url=configuration.get("gitremote", "push_url"),
web_url="",
path=".",
branch=configuration.get("gitremote", "push_branch", fallback="master"),
source=PackageSource.Local,
)
@staticmethod
def package_update(package: Package, target_dir: Path) -> str:
"""
clone specified package and update its content in cloned PKGBUILD repository
Args:
package(Package): built package to update pkgbuild repository
target_dir(Path): path to the cloned PKGBUILD repository
Returns:
str: relative path to be added as new file
"""
# instead of iterating by directory we can simplify the process
# firstly, we need to remove old data to make sure that removed files are not tracked anymore...
package_target_dir = target_dir / package.base
shutil.rmtree(package_target_dir, ignore_errors=True)
# ...secondly, we copy whole tree...
with TemporaryDirectory(ignore_cleanup_errors=True) as dir_name, (clone_dir := Path(dir_name)):
Sources.fetch(clone_dir, package.remote)
shutil.copytree(clone_dir, package_target_dir)
# ...and last, but not least, we remove the dot-git directory...
shutil.rmtree(package_target_dir / ".git", ignore_errors=True)
# ...and finally return path to the copied directory
return package.base
@staticmethod
def packages_update(result: Result, target_dir: Path) -> Generator[str, None, None]:
"""
update all packages from the build result
Args:
result(Result): build result
target_dir(Path): path to the cloned PKGBUILD repository
Yields:
str: path to updated files
"""
for package in result.success:
yield RemotePushTrigger.package_update(package, target_dir)
def on_result(self, result: Result, packages: Iterable[Package]) -> None:
"""
trigger action which will be called after build process with process result
Args:
result(Result): build result
packages(Iterable[Package]): list of all available packages
"""
with TemporaryDirectory(ignore_cleanup_errors=True) as dir_name, (clone_dir := Path(dir_name)):
Sources.fetch(clone_dir, self.remote_source)
Sources.push(clone_dir, self.remote_source, *RemotePushTrigger.packages_update(result, clone_dir))

1
src/ahriman/core/report/__init__.py
View File

@ -17,5 +17,4 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from ahriman.core.report.report import Report
from ahriman.core.report.report_trigger import ReportTrigger

2
src/ahriman/core/report/console.py
View File

@ -21,7 +21,7 @@ from typing import Iterable
from ahriman.core.configuration import Configuration
from ahriman.core.formatters import BuildPrinter
from ahriman.core.report import Report
from ahriman.core.report.report import Report
from ahriman.models.package import Package
from ahriman.models.result import Result

2
src/ahriman/core/report/email.py
View File

@ -25,8 +25,8 @@ from email.mime.text import MIMEText
from typing import Dict, Iterable
from ahriman.core.configuration import Configuration
from ahriman.core.report import Report
from ahriman.core.report.jinja_template import JinjaTemplate
from ahriman.core.report.report import Report
from ahriman.core.util import pretty_datetime
from ahriman.models.package import Package
from ahriman.models.result import Result

2
src/ahriman/core/report/html.py
View File

@ -20,8 +20,8 @@
from typing import Iterable
from ahriman.core.configuration import Configuration
from ahriman.core.report import Report
from ahriman.core.report.jinja_template import JinjaTemplate
from ahriman.core.report.report import Report
from ahriman.models.package import Package
from ahriman.models.result import Result

2
src/ahriman/core/report/report_trigger.py
View File

@ -21,7 +21,7 @@ from typing import Iterable
from ahriman.core.configuration import Configuration
from ahriman.core.triggers import Trigger
from ahriman.core.report import Report
from ahriman.core.report.report import Report
from ahriman.models.package import Package
from ahriman.models.result import Result

2
src/ahriman/core/report/telegram.py
View File

@ -23,8 +23,8 @@ import requests
from typing import Iterable
from ahriman.core.configuration import Configuration
from ahriman.core.report import Report
from ahriman.core.report.jinja_template import JinjaTemplate
from ahriman.core.report.report import Report
from ahriman.core.util import exception_response_text
from ahriman.models.package import Package
from ahriman.models.result import Result

12
src/ahriman/core/repository/executor.py
View File

@ -84,8 +84,7 @@ class Executor(Cleaner):
result = Result()
for single in updates:
with TemporaryDirectory(ignore_cleanup_errors=True) as dir_name, \
(build_dir := Path(dir_name)): # pylint: disable=confusing-with-statement
with TemporaryDirectory(ignore_cleanup_errors=True) as dir_name, (build_dir := Path(dir_name)):
try:
build_single(single, build_dir)
result.add_success(single)
@ -144,15 +143,6 @@ class Executor(Cleaner):
return self.repo.repo_path
def process_triggers(self, result: Result) -> None:
"""
process triggers setup by settings
Args:
result(Result): build result
"""
self.triggers.on_result(result, self.packages())
def process_update(self, packages: Iterable[Path]) -> Result:
"""
sign packages, add them to repository and update repository database

3
src/ahriman/core/tree.py
View File

@ -72,8 +72,7 @@ class Leaf:
Returns:
Leaf: loaded class
"""
with TemporaryDirectory(ignore_cleanup_errors=True) as dir_name, \
(clone_dir := Path(dir_name)): # pylint: disable=confusing-with-statement
with TemporaryDirectory(ignore_cleanup_errors=True) as dir_name, (clone_dir := Path(dir_name)):
Sources.load(clone_dir, package, database.patches_get(package.base), paths)
dependencies = Package.dependencies(clone_dir)
return cls(package, dependencies)

15
src/ahriman/core/triggers/trigger_loader.py
View File

@ -20,7 +20,6 @@
import contextlib
import importlib
import os
import weakref
from pathlib import Path
from types import ModuleType
@ -74,9 +73,15 @@ class TriggerLoader(LazyLogging):
self.load_trigger(trigger)
for trigger in configuration.getlist("build", "triggers")
]
self._on_stop_requested = False
self.on_start()
self._finalizer = weakref.finalize(self, self.on_stop)
def __del__(self) -> None:
"""
custom destructor object which calls on_stop in case if it was requested
"""
if not self._on_stop_requested:
return
self.on_stop()
@contextlib.contextmanager
def __execute_trigger(self, trigger: Trigger) -> Generator[None, None, None]:
@ -178,6 +183,7 @@ class TriggerLoader(LazyLogging):
result(Result): build result
packages(Iterable[Package]): list of all available packages
"""
self.logger.debug("executing triggers on result")
for trigger in self.triggers:
with self.__execute_trigger(trigger):
trigger.on_result(result, packages)
@ -186,6 +192,8 @@ class TriggerLoader(LazyLogging):
"""
run triggers on load
"""
self.logger.debug("executing triggers on start")
self._on_stop_requested = True
for trigger in self.triggers:
with self.__execute_trigger(trigger):
trigger.on_start()
@ -194,6 +202,7 @@ class TriggerLoader(LazyLogging):
"""
run triggers before the application exit
"""
self.logger.debug("executing triggers on stop")
for trigger in self.triggers:
with self.__execute_trigger(trigger):
trigger.on_stop()

1
src/ahriman/core/upload/__init__.py
View File

@ -17,5 +17,4 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
from ahriman.core.upload.upload import Upload
from ahriman.core.upload.upload_trigger import UploadTrigger

2
src/ahriman/core/upload/http_upload.py
View File

@ -24,7 +24,7 @@ from pathlib import Path
from typing import Any, Dict
from ahriman.core.configuration import Configuration
from ahriman.core.upload import Upload
from ahriman.core.upload.upload import Upload
from ahriman.core.util import exception_response_text

2
src/ahriman/core/upload/rsync.py
View File

@ -21,7 +21,7 @@ from pathlib import Path
from typing import Iterable
from ahriman.core.configuration import Configuration
from ahriman.core.upload import Upload
from ahriman.core.upload.upload import Upload
from ahriman.core.util import check_output
from ahriman.models.package import Package

2
src/ahriman/core/upload/s3.py
View File

@ -25,7 +25,7 @@ from pathlib import Path
from typing import Any, Dict, Iterable
from ahriman.core.configuration import Configuration
from ahriman.core.upload import Upload
from ahriman.core.upload.upload import Upload
from ahriman.core.util import walk
from ahriman.models.package import Package

2
src/ahriman/core/upload/upload_trigger.py
View File

@ -21,7 +21,7 @@ from typing import Iterable
from ahriman.core.configuration import Configuration
from ahriman.core.triggers import Trigger
from ahriman.core.upload import Upload
from ahriman.core.upload.upload import Upload
from ahriman.models.package import Package
from ahriman.models.result import Result

41
tests/ahriman/application/application/test_application.py
View File

@ -5,16 +5,6 @@ from ahriman.models.package import Package
from ahriman.models.result import Result
def test_finalize(application: Application, mocker: MockerFixture) -> None:
"""
must report and sync at the last
"""
triggers_mock = mocker.patch("ahriman.core.repository.Repository.process_triggers")
application._finalize(Result())
triggers_mock.assert_called_once_with(Result())
def test_known_packages(application: Application, package_ahriman: Package, mocker: MockerFixture) -> None:
"""
must return not empty list of known packages
@ -23,3 +13,34 @@ def test_known_packages(application: Application, package_ahriman: Package, mock
packages = application._known_packages()
assert len(packages) > 1
assert package_ahriman.base in packages
def test_on_result(application: Application, package_ahriman: Package, mocker: MockerFixture) -> None:
"""
must call on_result trigger function
"""
mocker.patch("ahriman.core.repository.Repository.packages", return_value=[package_ahriman])
triggers_mock = mocker.patch("ahriman.core.triggers.TriggerLoader.on_result")
application.on_result(Result())
triggers_mock.assert_called_once_with(Result(), [package_ahriman])
def test_on_start(application: Application, mocker: MockerFixture) -> None:
"""
must call on_start trigger function
"""
triggers_mock = mocker.patch("ahriman.core.triggers.TriggerLoader.on_start")
application.on_start()
triggers_mock.assert_called_once_with()
def test_on_stop(application: Application, mocker: MockerFixture) -> None:
"""
must call on_stop trigger function
"""
triggers_mock = mocker.patch("ahriman.core.triggers.TriggerLoader.on_stop")
application.on_stop()
triggers_mock.assert_called_once_with()

8
tests/ahriman/application/application/test_application_packages.py
View File

@ -11,12 +11,12 @@ from ahriman.models.package_source import PackageSource
from ahriman.models.result import Result
def test_finalize(application_packages: ApplicationPackages) -> None:
def test_on_result(application_packages: ApplicationPackages) -> None:
"""
must raise NotImplemented for missing finalize method
"""
with pytest.raises(NotImplementedError):
application_packages._finalize([])
application_packages.on_result(Result())
def test_known_packages(application_packages: ApplicationPackages) -> None:
@ -242,8 +242,8 @@ def test_remove(application_packages: ApplicationPackages, mocker: MockerFixture
must remove package
"""
executor_mock = mocker.patch("ahriman.core.repository.executor.Executor.process_remove")
finalize_mock = mocker.patch("ahriman.application.application.application_packages.ApplicationPackages._finalize")
on_result_mock = mocker.patch("ahriman.application.application.application_packages.ApplicationPackages.on_result")
application_packages.remove([])
executor_mock.assert_called_once_with([])
finalize_mock.assert_called_once_with(Result())
on_result_mock.assert_called_once_with(Result())

24
tests/ahriman/application/application/test_application_repository.py
View File

@ -9,12 +9,12 @@ from ahriman.models.package import Package
from ahriman.models.result import Result
def test_finalize(application_repository: ApplicationRepository) -> None:
def test_on_result(application_repository: ApplicationRepository) -> None:
"""
must raise NotImplemented for missing finalize method
"""
with pytest.raises(NotImplementedError):
application_repository._finalize(Result())
application_repository.on_result(Result())
def test_clean_cache(application_repository: ApplicationRepository, mocker: MockerFixture) -> None:
@ -63,8 +63,8 @@ def test_sign(application_repository: ApplicationRepository, package_ahriman: Pa
copy_mock = mocker.patch("shutil.copy")
update_mock = mocker.patch("ahriman.application.application.application_repository.ApplicationRepository.update")
sign_repository_mock = mocker.patch("ahriman.core.sign.gpg.GPG.process_sign_repository")
finalize_mock = mocker.patch(
"ahriman.application.application.application_repository.ApplicationRepository._finalize")
on_result_mock = mocker.patch(
"ahriman.application.application.application_repository.ApplicationRepository.on_result")
application_repository.sign([])
copy_mock.assert_has_calls([
@ -73,7 +73,7 @@ def test_sign(application_repository: ApplicationRepository, package_ahriman: Pa
])
update_mock.assert_called_once_with([])
sign_repository_mock.assert_called_once_with(application_repository.repository.repo.repo_path)
finalize_mock.assert_called_once_with(Result())
on_result_mock.assert_called_once_with(Result())
def test_sign_skip(application_repository: ApplicationRepository, package_ahriman: Package,
@ -84,7 +84,7 @@ def test_sign_skip(application_repository: ApplicationRepository, package_ahrima
package_ahriman.packages[package_ahriman.base].filename = None
mocker.patch("ahriman.core.repository.repository.Repository.packages", return_value=[package_ahriman])
mocker.patch("ahriman.application.application.application_repository.ApplicationRepository.update")
mocker.patch("ahriman.application.application.application_repository.ApplicationRepository._finalize")
mocker.patch("ahriman.application.application.application_repository.ApplicationRepository.on_result")
application_repository.sign([])
@ -99,8 +99,8 @@ def test_sign_specific(application_repository: ApplicationRepository, package_ah
copy_mock = mocker.patch("shutil.copy")
update_mock = mocker.patch("ahriman.application.application.application_repository.ApplicationRepository.update")
sign_repository_mock = mocker.patch("ahriman.core.sign.gpg.GPG.process_sign_repository")
finalize_mock = mocker.patch(
"ahriman.application.application.application_repository.ApplicationRepository._finalize")
on_result_mock = mocker.patch(
"ahriman.application.application.application_repository.ApplicationRepository.on_result")
filename = package_ahriman.packages[package_ahriman.base].filepath
application_repository.sign([package_ahriman.base])
@ -109,7 +109,7 @@ def test_sign_specific(application_repository: ApplicationRepository, package_ah
application_repository.repository.paths.packages / filename.name)
update_mock.assert_called_once_with([])
sign_repository_mock.assert_called_once_with(application_repository.repository.repo.repo_path)
finalize_mock.assert_called_once_with(Result())
on_result_mock.assert_called_once_with(Result())
def test_unknown_no_aur(application_repository: ApplicationRepository, package_ahriman: Package,
@ -163,13 +163,13 @@ def test_update(application_repository: ApplicationRepository, package_ahriman:
mocker.patch("ahriman.core.repository.repository.Repository.packages_built", return_value=paths)
build_mock = mocker.patch("ahriman.core.repository.executor.Executor.process_build", return_value=result)
update_mock = mocker.patch("ahriman.core.repository.executor.Executor.process_update", return_value=result)
finalize_mock = mocker.patch(
"ahriman.application.application.application_repository.ApplicationRepository._finalize")
on_result_mock = mocker.patch(
"ahriman.application.application.application_repository.ApplicationRepository.on_result")
application_repository.update([package_ahriman])
build_mock.assert_called_once_with([package_ahriman])
update_mock.assert_has_calls([mock.call(paths), mock.call(paths)])
finalize_mock.assert_has_calls([mock.call(result), mock.call(result)])
on_result_mock.assert_has_calls([mock.call(result), mock.call(result)])
def test_update_empty(application_repository: ApplicationRepository, package_ahriman: Package,

2
tests/ahriman/application/handlers/test_handler_add.py
View File

@ -35,9 +35,11 @@ def test_run(args: argparse.Namespace, configuration: Configuration, mocker: Moc
args = _default_args(args)
mocker.patch("ahriman.models.repository_paths.RepositoryPaths.tree_create")
application_mock = mocker.patch("ahriman.application.application.Application.add")
on_start_mock = mocker.patch("ahriman.application.application.Application.on_start")
Add.run(args, "x86_64", configuration, True, False)
application_mock.assert_called_once_with(args.package, args.source, args.without_dependencies)
on_start_mock.assert_called_once_with()
def test_run_with_updates(args: argparse.Namespace, configuration: Configuration,

2
tests/ahriman/application/handlers/test_handler_clean.py
View File

@ -30,6 +30,8 @@ def test_run(args: argparse.Namespace, configuration: Configuration, mocker: Moc
args = _default_args(args)
mocker.patch("ahriman.models.repository_paths.RepositoryPaths.tree_create")
application_mock = mocker.patch("ahriman.application.application.Application.clean")
on_start_mock = mocker.patch("ahriman.application.application.Application.on_start")
Clean.run(args, "x86_64", configuration, True, False)
application_mock.assert_called_once_with(False, False, False, False)
on_start_mock.assert_called_once_with()

2
tests/ahriman/application/handlers/test_handler_patch.py
View File

@ -36,9 +36,11 @@ def test_run(args: argparse.Namespace, configuration: Configuration, mocker: Moc
args.action = Action.Update
mocker.patch("ahriman.models.repository_paths.RepositoryPaths.tree_create")
application_mock = mocker.patch("ahriman.application.handlers.Patch.patch_set_create")
on_start_mock = mocker.patch("ahriman.application.application.Application.on_start")
Patch.run(args, "x86_64", configuration, True, False)
application_mock.assert_called_once_with(pytest.helpers.anyvar(int), Path(args.package), args.track)
on_start_mock.assert_called_once_with()
def test_run_list(args: argparse.Namespace, configuration: Configuration, mocker: MockerFixture) -> None:

2
tests/ahriman/application/handlers/test_handler_rebuild.py
View File

@ -41,11 +41,13 @@ def test_run(args: argparse.Namespace, package_ahriman: Package,
return_value=[package_ahriman])
application_mock = mocker.patch("ahriman.application.application.Application.update", return_value=result)
check_mock = mocker.patch("ahriman.application.handlers.Handler.check_if_empty")
on_start_mock = mocker.patch("ahriman.application.application.Application.on_start")
Rebuild.run(args, "x86_64", configuration, True, False)
application_packages_mock.assert_called_once_with(None)
application_mock.assert_called_once_with([package_ahriman])
check_mock.assert_has_calls([mock.call(False, False), mock.call(False, False)])
on_start_mock.assert_called_once_with()
def test_run_extract_packages(args: argparse.Namespace, configuration: Configuration, mocker: MockerFixture) -> None:

2
tests/ahriman/application/handlers/test_handler_remove.py
View File

@ -27,6 +27,8 @@ def test_run(args: argparse.Namespace, configuration: Configuration, mocker: Moc
args = _default_args(args)
mocker.patch("ahriman.models.repository_paths.RepositoryPaths.tree_create")
application_mock = mocker.patch("ahriman.application.application.Application.remove")
on_start_mock = mocker.patch("ahriman.application.application.Application.on_start")
Remove.run(args, "x86_64", configuration, True, False)
application_mock.assert_called_once_with([])
on_start_mock.assert_called_once_with()

2
tests/ahriman/application/handlers/test_handler_remove_unknown.py
View File

@ -32,10 +32,12 @@ def test_run(args: argparse.Namespace, package_ahriman: Package,
application_mock = mocker.patch("ahriman.application.application.Application.unknown",
return_value=[package_ahriman])
remove_mock = mocker.patch("ahriman.application.application.Application.remove")
on_start_mock = mocker.patch("ahriman.application.application.Application.on_start")
RemoveUnknown.run(args, "x86_64", configuration, True, False)
application_mock.assert_called_once_with()
remove_mock.assert_called_once_with([package_ahriman])
on_start_mock.assert_called_once_with()
def test_run_dry_run(args: argparse.Namespace, configuration: Configuration, package_ahriman: Package,

4
tests/ahriman/application/handlers/test_handler_triggers.py
View File

@ -28,10 +28,12 @@ def test_run(args: argparse.Namespace, configuration: Configuration, mocker: Moc
"""
args = _default_args(args)
mocker.patch("ahriman.models.repository_paths.RepositoryPaths.tree_create")
application_mock = mocker.patch("ahriman.core.repository.Repository.process_triggers")
application_mock = mocker.patch("ahriman.application.application.Application.on_result")
on_start_mock = mocker.patch("ahriman.application.application.Application.on_start")
Triggers.run(args, "x86_64", configuration, True, False)
application_mock.assert_called_once_with(Result())
on_start_mock.assert_called_once_with()
def test_run_trigger(args: argparse.Namespace, configuration: Configuration, package_ahriman: Package,

2
tests/ahriman/application/handlers/test_handler_update.py
View File

@ -43,12 +43,14 @@ def test_run(args: argparse.Namespace, package_ahriman: Package,
application_mock = mocker.patch("ahriman.application.application.Application.update", return_value=result)
check_mock = mocker.patch("ahriman.application.handlers.Handler.check_if_empty")
updates_mock = mocker.patch("ahriman.application.application.Application.updates", return_value=[package_ahriman])
on_start_mock = mocker.patch("ahriman.application.application.Application.on_start")
Update.run(args, "x86_64", configuration, True, False)
application_mock.assert_called_once_with([package_ahriman])
updates_mock.assert_called_once_with(args.package, args.no_aur, args.no_local, args.no_manual, args.no_vcs,
pytest.helpers.anyvar(int))
check_mock.assert_has_calls([mock.call(False, False), mock.call(False, False)])
on_start_mock.assert_called_once_with()
def test_run_empty_exception(args: argparse.Namespace, configuration: Configuration, mocker: MockerFixture) -> None:

46
tests/ahriman/core/build_tools/test_sources.py
View File

@ -223,6 +223,23 @@ def test_patch_create_with_newline(mocker: MockerFixture) -> None:
assert Sources.patch_create(Path("local"), "glob").endswith("\n")
def test_push(package_ahriman: Package, mocker: MockerFixture) -> None:
"""
must correctly push files to remote repository
"""
add_mock = mocker.patch("ahriman.core.build_tools.sources.Sources.add")
commit_mock = mocker.patch("ahriman.core.build_tools.sources.Sources.commit")
check_output_mock = mocker.patch("ahriman.core.build_tools.sources.Sources._check_output")
local = Path("local")
Sources.push(Path("local"), package_ahriman.remote, "glob")
add_mock.assert_called_once_with(local, "glob")
commit_mock.assert_called_once_with(local)
check_output_mock.assert_called_once_with(
"git", "push", package_ahriman.remote.git_url, package_ahriman.remote.branch,
exception=None, cwd=local, logger=pytest.helpers.anyvar(int))
def test_add(sources: Sources, mocker: MockerFixture) -> None:
"""
must add files to git
@ -249,6 +266,35 @@ def test_add_skip(sources: Sources, mocker: MockerFixture) -> None:
check_output_mock.assert_not_called()
def test_commit(sources: Sources, mocker: MockerFixture) -> None:
"""
must commit changes
"""
check_output_mock = mocker.patch("ahriman.core.build_tools.sources.Sources._check_output")
local = Path("local")
commit_message = "Commit message"
sources.commit(local, commit_message=commit_message)
check_output_mock.assert_called_once_with(
"git", "commit", "--all", "--message", commit_message,
exception=None, cwd=local, logger=pytest.helpers.anyvar(int)
)
def test_commit_autogenerated(sources: Sources, mocker: MockerFixture) -> None:
"""
must commit changes with autogenerated commit message
"""
check_output_mock = mocker.patch("ahriman.core.build_tools.sources.Sources._check_output")
local = Path("local")
sources.commit(Path("local"))
check_output_mock.assert_called_once_with(
"git", "commit", "--all", "--message", pytest.helpers.anyvar(str, strict=True),
exception=None, cwd=local, logger=pytest.helpers.anyvar(int)
)
def test_diff(sources: Sources, mocker: MockerFixture) -> None:
"""
must calculate diff

58
tests/ahriman/core/gitremote/test_remote_pull_trigger.py Normal file
View File

@ -0,0 +1,58 @@
import pytest
from pathlib import Path
from pytest_mock import MockerFixture
from unittest import mock
from ahriman.core.configuration import Configuration
from ahriman.core.gitremote import RemotePullTrigger
def test_on_start(configuration: Configuration, mocker: MockerFixture) -> None:
"""
must clone repo on start
"""
clone_mock = mocker.patch("ahriman.core.gitremote.RemotePullTrigger.repo_clone")
trigger = RemotePullTrigger("x86_64", configuration)
trigger.on_start()
clone_mock.assert_called_once_with()
def test_repo_clone(configuration: Configuration, mocker: MockerFixture) -> None:
"""
must clone repository locally and copy its content
"""
fetch_mock = mocker.patch("ahriman.core.build_tools.sources.Sources.fetch")
copy_mock = mocker.patch("ahriman.core.gitremote.RemotePullTrigger.repo_copy")
trigger = RemotePullTrigger("x86_64", configuration)
trigger.repo_clone()
fetch_mock.assert_called_once_with(pytest.helpers.anyvar(int), trigger.remote_source)
copy_mock.assert_called_once_with(pytest.helpers.anyvar(int))
def test_repo_copy(configuration: Configuration, mocker: MockerFixture) -> None:
"""
must copy repository tree from temporary directory to the local cache
"""
mocker.patch("ahriman.core.gitremote.remote_pull_trigger.walk", return_value=[
Path("local") / "package1" / "PKGBUILD",
Path("local") / "package1" / ".SRCINFO",
Path("local") / "package2" / ".SRCINFO",
Path("local") / "package3" / "PKGBUILD",
Path("local") / "package3" / ".SRCINFO",
])
copytree_mock = mocker.patch("shutil.copytree")
init_mock = mocker.patch("ahriman.core.build_tools.sources.Sources.init")
trigger = RemotePullTrigger("x86_64", configuration)
trigger.repo_copy(Path("local"))
copytree_mock.assert_has_calls([
mock.call(Path("local") / "package1", configuration.repository_paths.cache_for("package1"), dirs_exist_ok=True),
mock.call(Path("local") / "package3", configuration.repository_paths.cache_for("package3"), dirs_exist_ok=True),
])
init_mock.assert_has_calls([
mock.call(configuration.repository_paths.cache_for("package1")),
mock.call(configuration.repository_paths.cache_for("package3")),
])

56
tests/ahriman/core/gitremote/test_remote_push_trigger.py Normal file
View File

@ -0,0 +1,56 @@
import pytest
from pathlib import Path
from pytest_mock import MockerFixture
from unittest import mock
from ahriman.core.configuration import Configuration
from ahriman.core.gitremote import RemotePushTrigger
from ahriman.models.package import Package
from ahriman.models.result import Result
def test_package_update(package_ahriman: Package, mocker: MockerFixture) -> None:
"""
must update single package
"""
rmtree_mock = mocker.patch("shutil.rmtree")
fetch_mock = mocker.patch("ahriman.core.build_tools.sources.Sources.fetch")
copytree_mock = mocker.patch("shutil.copytree")
local = Path("local")
RemotePushTrigger.package_update(package_ahriman, local)
rmtree_mock.assert_has_calls([
mock.call(local / package_ahriman.base, ignore_errors=True),
mock.call(pytest.helpers.anyvar(int), onerror=pytest.helpers.anyvar(int)), # removal of the TemporaryDirectory
mock.call(local / package_ahriman.base / ".git", ignore_errors=True),
])
fetch_mock.assert_called_once_with(pytest.helpers.anyvar(int), package_ahriman.remote)
copytree_mock.assert_called_once_with(pytest.helpers.anyvar(int), local / package_ahriman.base)
def test_packages_update(result: Result, package_ahriman: Package, mocker: MockerFixture) -> None:
"""
must generate packages update
"""
update_mock = mocker.patch("ahriman.core.gitremote.RemotePushTrigger.package_update",
return_value=[package_ahriman.base])
local = Path("local")
assert list(RemotePushTrigger.packages_update(result, local))
update_mock.assert_called_once_with(package_ahriman, local)
def test_on_result(configuration: Configuration, result: Result, package_ahriman: Package,
mocker: MockerFixture) -> None:
"""
must push changes on result
"""
mocker.patch("ahriman.core.gitremote.RemotePushTrigger.packages_update", return_value=[package_ahriman.base])
fetch_mock = mocker.patch("ahriman.core.build_tools.sources.Sources.fetch")
push_mock = mocker.patch("ahriman.core.build_tools.sources.Sources.push")
trigger = RemotePushTrigger("x86_64", configuration)
trigger.on_result(result, [package_ahriman])
fetch_mock.assert_called_once_with(pytest.helpers.anyvar(int), trigger.remote_source)
push_mock.assert_called_once_with(pytest.helpers.anyvar(int), trigger.remote_source, package_ahriman.base)

4
tests/ahriman/core/report/test_report.py
View File

@ -4,7 +4,7 @@ from pytest_mock import MockerFixture
from ahriman.core.configuration import Configuration
from ahriman.core.exceptions import ReportFailed
from ahriman.core.report import Report
from ahriman.core.report.report import Report
from ahriman.models.report_settings import ReportSettings
from ahriman.models.result import Result
@ -23,7 +23,7 @@ def test_report_dummy(configuration: Configuration, result: Result, mocker: Mock
must construct dummy report class
"""
mocker.patch("ahriman.models.report_settings.ReportSettings.from_option", return_value=ReportSettings.Disabled)
report_mock = mocker.patch("ahriman.core.report.Report.generate")
report_mock = mocker.patch("ahriman.core.report.report.Report.generate")
Report.load("x86_64", configuration, "disabled").run(result, [])
report_mock.assert_called_once_with([], result)

2
tests/ahriman/core/report/test_report_trigger.py
View File

@ -10,7 +10,7 @@ def test_on_result(configuration: Configuration, mocker: MockerFixture) -> None:
must run report for specified targets
"""
configuration.set_option("report", "target", "email")
run_mock = mocker.patch("ahriman.core.report.Report.run")
run_mock = mocker.patch("ahriman.core.report.report.Report.run")
trigger = ReportTrigger("x86_64", configuration)
trigger.on_result(Result(), [])

13
tests/ahriman/core/repository/test_executor.py
View File

@ -4,10 +4,8 @@ from pathlib import Path
from pytest_mock import MockerFixture
from unittest import mock
from ahriman.core.report import Report
from ahriman.core.repository.executor import Executor
from ahriman.models.package import Package
from ahriman.models.result import Result
def test_load_archives(executor: Executor) -> None:
@ -144,17 +142,6 @@ def test_process_remove_nothing(executor: Executor, package_ahriman: Package, pa
repo_remove_mock.assert_not_called()
def test_process_triggers(executor: Executor, package_ahriman: Package, result: Result, mocker: MockerFixture) -> None:
"""
must process report
"""
mocker.patch("ahriman.core.repository.executor.Executor.packages", return_value=[package_ahriman])
triggers_mock = mocker.patch("ahriman.core.triggers.TriggerLoader.on_result")
executor.process_triggers(result)
triggers_mock.assert_called_once_with(result, [package_ahriman])
def test_process_update(executor: Executor, package_ahriman: Package, mocker: MockerFixture) -> None:
"""
must run update process

37
tests/ahriman/core/triggers/test_trigger_loader.py
View File

@ -10,19 +10,6 @@ from ahriman.models.package import Package
from ahriman.models.result import Result
def test_init_at_exit(configuration: Configuration, mocker: MockerFixture) -> None:
"""
must call on_start on init and on_stop on exit
"""
on_start_mock = mocker.patch("ahriman.core.triggers.trigger_loader.TriggerLoader.on_start")
on_stop_mock = mocker.patch("ahriman.core.triggers.trigger_loader.TriggerLoader.on_stop")
trigger_loader = TriggerLoader("x86_64", configuration)
on_start_mock.assert_called_once_with()
del trigger_loader
on_stop_mock.assert_called_once_with()
def test_load_trigger_package(trigger_loader: TriggerLoader) -> None:
"""
must load trigger from package
@ -123,10 +110,34 @@ def test_on_start(trigger_loader: TriggerLoader, package_ahriman: Package, mocke
report_mock = mocker.patch("ahriman.core.report.ReportTrigger.on_start")
trigger_loader.on_start()
assert trigger_loader._on_stop_requested
report_mock.assert_called_once_with()
upload_mock.assert_called_once_with()
def test_on_stop_with_on_start(configuration: Configuration, mocker: MockerFixture) -> None:
"""
must call on_stop on exit if on_start was called
"""
on_stop_mock = mocker.patch("ahriman.core.triggers.trigger_loader.TriggerLoader.on_stop")
trigger_loader = TriggerLoader("x86_64", configuration)
trigger_loader.on_start()
del trigger_loader
on_stop_mock.assert_called_once_with()
def test_on_stop_without_on_start(configuration: Configuration, mocker: MockerFixture) -> None:
"""
must call not on_stop on exit if on_start wasn't called
"""
on_stop_mock = mocker.patch("ahriman.core.triggers.trigger_loader.TriggerLoader.on_stop")
trigger_loader = TriggerLoader("x86_64", configuration)
del trigger_loader
on_stop_mock.assert_not_called()
def test_on_stop(trigger_loader: TriggerLoader, package_ahriman: Package, mocker: MockerFixture) -> None:
"""
must run triggers on stop

4
tests/ahriman/core/upload/test_upload.py
View File

@ -5,7 +5,7 @@ from pytest_mock import MockerFixture
from ahriman.core.configuration import Configuration
from ahriman.core.exceptions import SyncFailed
from ahriman.core.upload import Upload
from ahriman.core.upload.upload import Upload
from ahriman.models.upload_settings import UploadSettings
@ -23,7 +23,7 @@ def test_report_dummy(configuration: Configuration, mocker: MockerFixture) -> No
must construct dummy upload class
"""
mocker.patch("ahriman.models.upload_settings.UploadSettings.from_option", return_value=UploadSettings.Disabled)
upload_mock = mocker.patch("ahriman.core.upload.Upload.sync")
upload_mock = mocker.patch("ahriman.core.upload.upload.Upload.sync")
Upload.load("x86_64", configuration, "disabled").run(Path("path"), [])
upload_mock.assert_called_once_with(Path("path"), [])

2
tests/ahriman/core/upload/test_upload_trigger.py
View File

@ -10,7 +10,7 @@ def test_on_result(configuration: Configuration, mocker: MockerFixture) -> None:
must run report for specified targets
"""
configuration.set_option("upload", "target", "rsync")
run_mock = mocker.patch("ahriman.core.upload.Upload.run")
run_mock = mocker.patch("ahriman.core.upload.upload.Upload.run")
trigger = UploadTrigger("x86_64", configuration)
trigger.on_result(Result(), [])

4
tests/testresources/core/ahriman.ini
View File

@ -31,6 +31,10 @@ root = ../../../
[sign]
target =
[gitremote]
pull_url = https://github.com/arcan1s/repository.git
push_url = https://github.com/arcan1s/repository.git
[report]
target =