Release 2.3.0rc3

Old logic used OR condition, i.e. if set from-database, it would ignore
the --depends-on flag. In new logic it calculates dependencies based on
the package list, which can be retrieved from database

.pylintrc

@@ -149,7 +149,7 @@ indent-string='    '
 max-line-length=100
 
 # Maximum number of lines in a module.
-max-module-lines=1000
+max-module-lines=400
 
 # Allow the body of a class to be on the same line as the declaration if body
 # contains single statement.

CONTRIBUTING.md

@@ -1,6 +1,6 @@
 # Contributing to ahriman
 
-Welcome to ahriman! The goal of the project is to provide the best user experience to manage Archlinux repositories. In order to follow this we set some limitations for the issue creations and heavily restricted code contribution.
+Welcome to ahriman! The goal of the project is to provide the best user experience to manage Arch linux repositories. In order to follow this we set some limitations for the issue creations and heavily restricted code contribution.
 
 ## Create an issue
 
@@ -26,12 +26,13 @@ In order to resolve all difficult cases the `autopep8` is used. You can perform
 Again, the most checks can be performed by `make check` command, though some additional guidelines must be applied:
 
 * Every class, every function (including private and protected), every attribute must be documented. The project follows [Google style documentation](https://google.github.io/styleguide/pyguide.html). The only exception is local functions.
+* Type annotations are the must, even for local functions.
 * For any path interactions `pathlib.Path` must be used.
 * Configuration interactions must go through `ahriman.core.configuration.Configuration` class instance.
 * In case if class load requires some actions, it is recommended to create class method which can be used for class instantiating.
 * The code must follow the exception safety, unless it is explicitly asked by end user. It means that most exceptions must be handled and printed to log, no other actions must be done (e.g. raising another exception).
 * For the external command `ahriman.core.util.check_output` function must be used.
-* Every temporary file/directory must be removed at the end of processing, no matter what. The ``tempfile`` module provides good ways to do it.
+* Every temporary file/directory must be removed at the end of processing, no matter what. The `tempfile` module provides good ways to do it.
 * Import order must be the following:
 
     ```python
@@ -55,10 +56,10 @@ Again, the most checks can be performed by `make check` command, though some add
     from ahriman.core.configuration import Configuration
     ```
 
-* One file should define only one class, exception is class satellites in case if file length remain less than 200 lines.
+* One file should define only one class, exception is class satellites in case if file length remains less than 400 lines.
 * It is possible to create file which contains some functions (e.g. `ahriman.core.util`), but in this case you would need to define `__all__` attribute.
-* The file size mentioned above must be applicable in general. In case of big classes consider splitting them into traits.
-* No global variable allowed outside of `ahriman.version` module.
+* The file size mentioned above must be applicable in general. In case of big classes consider splitting them into traits. Note, however, that `pylint` includes comments and docstrings into counter, thus you need to check file size by other tools.
+* No global variable is allowed outside of `ahriman.version` module.
 * Single quotes are not allowed. The reason behind this restriction is the fact that docstrings must be written by using double quotes only, and we would like to make style consistent.
 * If your class writes anything to log, the `ahriman.core.lazy_logging.LazyLogging` trait must be used.
 

docs/ahriman.1

@@ -1,4 +1,4 @@
-.TH AHRIMAN "1" "2022\-11\-10" "ahriman" "Generated Python Manual"
+.TH AHRIMAN "1" "2022\-11\-11" "ahriman" "Generated Python Manual"
 .SH NAME
 ahriman
 .SH SYNOPSIS
@@ -154,7 +154,7 @@ search for package in AUR using API
 
 .TP
 \fBsearch\fR
-search terms, can be specified multiple times, result will match all terms
+search terms, can be specified multiple times, the result will match all terms
 
 .SH OPTIONS \fI\,'ahriman aur\-search'\/\fR
 .TP
@@ -309,7 +309,7 @@ remove the package from the status page
 
 .TP
 \fBpackage\fR
-remove specified packages
+remove specified packages from status page
 
 .SH COMMAND \fI\,'ahriman package\-status\-update'\/\fR
 usage: ahriman package\-status\-update [\-h] [\-s {unknown,pending,building,failed,success}] [package ...]
@@ -323,7 +323,7 @@ set status for specified packages. If no packages supplied, service status will
 .SH OPTIONS \fI\,'ahriman package\-status\-update'\/\fR
 .TP
 \fB\-s\fR \fI\,{unknown,pending,building,failed,success}\/\fR, \fB\-\-status\fR \fI\,{unknown,pending,building,failed,success}\/\fR
-new status
+new package build status
 
 .SH COMMAND \fI\,'ahriman patch\-add'\/\fR
 usage: ahriman patch\-add [\-h] package variable [patch]
@@ -392,7 +392,7 @@ files which has to be tracked
 .SH COMMAND \fI\,'ahriman repo\-backup'\/\fR
 usage: ahriman repo\-backup [\-h] path
 
-backup settings and database
+backup repository settings and database
 
 .TP
 \fBpath\fR
@@ -401,7 +401,7 @@ path of the output archive
 .SH COMMAND \fI\,'ahriman repo\-check'\/\fR
 usage: ahriman repo\-check [\-h] [\-e] [\-\-vcs | \-\-no\-vcs] [\-y] [package ...]
 
-check for packages updates. Same as update \-\-dry\-run \-\-no\-manual
+check for packages updates. Same as repo\-update \-\-dry\-run \-\-no\-manual
 
 .TP
 \fBpackage\fR
@@ -460,7 +460,7 @@ force rebuild whole repository
 .SH OPTIONS \fI\,'ahriman repo\-rebuild'\/\fR
 .TP
 \fB\-\-depends\-on\fR \fI\,DEPENDS_ON\/\fR
-only rebuild packages that depend on specified package
+only rebuild packages that depend on specified packages
 
 .TP
 \fB\-\-dry\-run\fR

package/archlinux/PKGBUILD

@@ -1,7 +1,7 @@
 # Maintainer: Evgeniy Alekseev
 
 pkgname='ahriman'
-pkgver=2.2.2
+pkgver=2.3.0rc3
 pkgrel=1
 pkgdesc="ArcH linux ReposItory MANager"
 arch=('any')

src/ahriman/application/ahriman.py

@@ -17,6 +17,7 @@
 # You should have received a copy of the GNU General Public License
 # along with this program. If not, see <http://www.gnu.org/licenses/>.
 #
+# pylint: disable=too-many-lines
 import argparse
 import sys
 import tempfile
@@ -132,7 +133,7 @@ def _set_aur_search_parser(root: SubParserAction) -> argparse.ArgumentParser:
     """
     parser = root.add_parser("aur-search", aliases=["search"], help="search for package",
                              description="search for package in AUR using API", formatter_class=_formatter)
-    parser.add_argument("search", help="search terms, can be specified multiple times, result will match all terms",
+    parser.add_argument("search", help="search terms, can be specified multiple times, the result will match all terms",
                         nargs="+")
     parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty", action="store_true")
     parser.add_argument("--info", help="show additional package information",
@@ -326,7 +327,7 @@ def _set_package_status_remove_parser(root: SubParserAction) -> argparse.Argumen
                              epilog="Please note that this subcommand does not remove the package itself, it just "
                                     "clears the status page.",
                              formatter_class=_formatter)
-    parser.add_argument("package", help="remove specified packages", nargs="+")
+    parser.add_argument("package", help="remove specified packages from status page", nargs="+")
     parser.set_defaults(handler=handlers.StatusUpdate, action=Action.Remove, lock=None, report=False, quiet=True,
                         unsafe=True)
     return parser
@@ -347,7 +348,7 @@ def _set_package_status_update_parser(root: SubParserAction) -> argparse.Argumen
     parser.add_argument("package", help="set status for specified packages. "
                                         "If no packages supplied, service status will be updated",
                         nargs="*")
-    parser.add_argument("-s", "--status", help="new status",
+    parser.add_argument("-s", "--status", help="new package build status",
                         type=BuildStatusEnum, choices=enum_values(BuildStatusEnum), default=BuildStatusEnum.Success)
     parser.set_defaults(handler=handlers.StatusUpdate, action=Action.Update, lock=None, report=False, quiet=True,
                         unsafe=True)
@@ -366,9 +367,9 @@ def _set_patch_add_parser(root: SubParserAction) -> argparse.ArgumentParser:
     """
     parser = root.add_parser("patch-add", help="add patch for PKGBUILD function",
                              description="create or update patched PKGBUILD function or variable",
-                             epilog="Unlike ``patch-set-add``, this function allows to patch only one PKGBUILD f"
-                                    "unction, e.g. typing ``ahriman patch-add ahriman version`` it will change the "
-                                    "``version`` inside PKGBUILD, typing ``ahriman patch-add ahriman build()`` "
+                             epilog="Unlike ``patch-set-add``, this function allows to patch only one PKGBUILD "
+                                    "function, e.g. typing ``ahriman patch-add ahriman pkgver`` it will change the "
+                                    "``pkgver`` inside PKGBUILD, typing ``ahriman patch-add ahriman build()`` "
                                     "it will change ``build()`` function inside PKGBUILD",
                              formatter_class=_formatter)
     parser.add_argument("package", help="package base")
@@ -457,7 +458,7 @@ def _set_repo_backup_parser(root: SubParserAction) -> argparse.ArgumentParser:
         argparse.ArgumentParser: created argument parser
     """
     parser = root.add_parser("repo-backup", help="backup repository data",
-                             description="backup settings and database", formatter_class=_formatter)
+                             description="backup repository settings and database", formatter_class=_formatter)
     parser.add_argument("path", help="path of the output archive", type=Path)
     parser.set_defaults(handler=handlers.Backup, architecture=[""], lock=None, report=False, unsafe=True)
     return parser
@@ -474,7 +475,7 @@ def _set_repo_check_parser(root: SubParserAction) -> argparse.ArgumentParser:
         argparse.ArgumentParser: created argument parser
     """
     parser = root.add_parser("repo-check", aliases=["check"], help="check for updates",
-                             description="check for packages updates. Same as update --dry-run --no-manual",
+                             description="check for packages updates. Same as repo-update --dry-run --no-manual",
                              formatter_class=_formatter)
     parser.add_argument("package", help="filter check by package base", nargs="*")
     parser.add_argument("-e", "--exit-code", help="return non-zero exit status if result is empty", action="store_true")
@@ -545,7 +546,7 @@ def _set_repo_rebuild_parser(root: SubParserAction) -> argparse.ArgumentParser:
     """
     parser = root.add_parser("repo-rebuild", aliases=["rebuild"], help="rebuild repository",
                              description="force rebuild whole repository", formatter_class=_formatter)
-    parser.add_argument("--depends-on", help="only rebuild packages that depend on specified package", action="append")
+    parser.add_argument("--depends-on", help="only rebuild packages that depend on specified packages", action="append")
     parser.add_argument("--dry-run", help="just perform check for packages without rebuild process itself",
                         action="store_true")
     parser.add_argument("--from-database",

src/ahriman/application/handlers/rebuild.py

@@ -51,10 +51,8 @@ class Rebuild(Handler):
         application = Application(architecture, configuration, report=report, unsafe=unsafe)
         application.on_start()
 
-        if args.from_database:
-            updates = Rebuild.extract_packages(application)
-        else:
-            updates = application.repository.packages_depend_on(depends_on)
+        packages = Rebuild.extract_packages(application, from_database=args.from_database)
+        updates = application.repository.packages_depend_on(packages, depends_on)
 
         Rebuild.check_if_empty(args.exit_code, not updates)
         if args.dry_run:
@@ -66,14 +64,17 @@ class Rebuild(Handler):
         Rebuild.check_if_empty(args.exit_code, result.is_empty)
 
     @staticmethod
-    def extract_packages(application: Application) -> List[Package]:
+    def extract_packages(application: Application, *, from_database: bool) -> List[Package]:
         """
         extract packages from database file
 
         Args:
             application(Application): application instance
+            from_database(bool): extract packages from database instead of repository filesystem
 
         Returns:
             List[Package]: list of packages which were stored in database
         """
+        if from_database:
+            return application.repository.packages()
         return [package for (package, _) in application.database.packages_get()]

src/ahriman/core/build_tools/sources.py

@@ -86,7 +86,7 @@ class Sources(LazyLogging):
         elif remote is not None:
             instance.logger.info("clone remote %s to %s using branch %s", remote.git_url, sources_dir, branch)
             Sources._check_output("git", "clone", "--branch", branch, "--single-branch",
-                                  remote.git_url, str(sources_dir), cwd=sources_dir, logger=instance.logger)
+                                  remote.git_url, str(sources_dir), cwd=sources_dir.parent, logger=instance.logger)
         else:
             # it will cause an exception later
             instance.logger.error("%s is not initialized, but no remote provided", sources_dir)

src/ahriman/core/repository/repository.py

@@ -99,17 +99,17 @@ class Repository(Executor, UpdateHandler):
         """
         return list(filter(package_like, self.paths.packages.iterdir()))
 
-    def packages_depend_on(self, depends_on: Optional[Iterable[str]]) -> List[Package]:
+    def packages_depend_on(self, packages: List[Package], depends_on: Optional[Iterable[str]]) -> List[Package]:
         """
         extract list of packages which depends on specified package
 
         Args:
+            packages(List[Package]): list of packages to be filtered
             depends_on(Optional[Iterable[str]]): dependencies of the packages
 
         Returns:
             List[Package]: list of repository packages which depend on specified packages
         """
-        packages = self.packages()
         if depends_on is None:
             return packages  # no list provided extract everything by default
         depends_on = set(depends_on)

src/ahriman/version.py

@@ -17,4 +17,4 @@
 # You should have received a copy of the GNU General Public License
 # along with this program. If not, see <http://www.gnu.org/licenses/>.
 #
-__version__ = "2.2.2"
+__version__ = "2.3.0rc3"

tests/ahriman/application/handlers/test_handler_rebuild.py

@@ -37,6 +37,7 @@ def test_run(args: argparse.Namespace, package_ahriman: Package,
     result = Result()
     result.add_success(package_ahriman)
     mocker.patch("ahriman.models.repository_paths.RepositoryPaths.tree_create")
+    extract_mock = mocker.patch("ahriman.application.handlers.Rebuild.extract_packages", return_value=[package_ahriman])
     application_packages_mock = mocker.patch("ahriman.core.repository.repository.Repository.packages_depend_on",
                                              return_value=[package_ahriman])
     application_mock = mocker.patch("ahriman.application.application.Application.update", return_value=result)
@@ -44,7 +45,8 @@ def test_run(args: argparse.Namespace, package_ahriman: Package,
     on_start_mock = mocker.patch("ahriman.application.application.Application.on_start")
 
     Rebuild.run(args, "x86_64", configuration, report=False, unsafe=False)
-    application_packages_mock.assert_called_once_with(None)
+    extract_mock.assert_called_once_with(pytest.helpers.anyvar(int), from_database=args.from_database)
+    application_packages_mock.assert_called_once_with([package_ahriman], None)
     application_mock.assert_called_once_with([package_ahriman])
     check_mock.assert_has_calls([MockCall(False, False), MockCall(False, False)])
     on_start_mock.assert_called_once_with()
@@ -62,7 +64,7 @@ def test_run_extract_packages(args: argparse.Namespace, configuration: Configura
     extract_mock = mocker.patch("ahriman.application.handlers.Rebuild.extract_packages", return_value=[])
 
     Rebuild.run(args, "x86_64", configuration, report=False, unsafe=False)
-    extract_mock.assert_called_once_with(pytest.helpers.anyvar(int))
+    extract_mock.assert_called_once_with(pytest.helpers.anyvar(int), from_database=args.from_database)
 
 
 def test_run_dry_run(args: argparse.Namespace, configuration: Configuration,
@@ -93,7 +95,7 @@ def test_run_filter(args: argparse.Namespace, configuration: Configuration, mock
     application_packages_mock = mocker.patch("ahriman.core.repository.repository.Repository.packages_depend_on")
 
     Rebuild.run(args, "x86_64", configuration, report=False, unsafe=False)
-    application_packages_mock.assert_called_once_with({"python-aur"})
+    application_packages_mock.assert_called_once_with([], {"python-aur"})
 
 
 def test_run_without_filter(args: argparse.Namespace, configuration: Configuration, mocker: MockerFixture) -> None:
@@ -106,7 +108,7 @@ def test_run_without_filter(args: argparse.Namespace, configuration: Configurati
     application_packages_mock = mocker.patch("ahriman.core.repository.repository.Repository.packages_depend_on")
 
     Rebuild.run(args, "x86_64", configuration, report=False, unsafe=False)
-    application_packages_mock.assert_called_once_with(None)
+    application_packages_mock.assert_called_once_with([], None)
 
 
 def test_run_update_empty_exception(args: argparse.Namespace, configuration: Configuration,
@@ -146,5 +148,14 @@ def test_extract_packages(application: Application, mocker: MockerFixture) -> No
     must extract packages from database
     """
     packages_mock = mocker.patch("ahriman.core.database.SQLite.packages_get")
-    Rebuild.extract_packages(application)
+    Rebuild.extract_packages(application, from_database=False)
+    packages_mock.assert_called_once_with()
+
+
+def test_extract_packages_from_database(application: Application, mocker: MockerFixture) -> None:
+    """
+    must extract packages from database
+    """
+    packages_mock = mocker.patch("ahriman.core.repository.repository.Repository.packages")
+    Rebuild.extract_packages(application, from_database=True)
     packages_mock.assert_called_once_with()

tests/ahriman/core/build_tools/test_sources.py

@@ -75,7 +75,7 @@ def test_fetch_new(remote_source: RemoteSource, mocker: MockerFixture) -> None:
     Sources.fetch(local, remote_source)
     check_output_mock.assert_has_calls([
         MockCall("git", "clone", "--branch", remote_source.branch, "--single-branch",
-                 remote_source.git_url, str(local), cwd=local, logger=pytest.helpers.anyvar(int)),
+                 remote_source.git_url, str(local), cwd=local.parent, logger=pytest.helpers.anyvar(int)),
         MockCall("git", "checkout", "--force", remote_source.branch, cwd=local, logger=pytest.helpers.anyvar(int)),
         MockCall("git", "reset", "--hard", f"origin/{remote_source.branch}",
                  cwd=local, logger=pytest.helpers.anyvar(int))

tests/ahriman/core/repository/test_repository.py

@@ -92,7 +92,7 @@ def test_packages_depend_on(repository: Repository, package_ahriman: Package, pa
     """
     mocker.patch("ahriman.core.repository.repository.Repository.packages",
                  return_value=[package_ahriman, package_python_schedule])
-    assert repository.packages_depend_on(["python-aur"]) == [package_ahriman]
+    assert repository.packages_depend_on([package_ahriman], ["python-aur"]) == [package_ahriman]
 
 
 def test_packages_depend_on_empty(repository: Repository, package_ahriman: Package, package_python_schedule: Package,
@@ -102,4 +102,5 @@ def test_packages_depend_on_empty(repository: Repository, package_ahriman: Packa
     """
     mocker.patch("ahriman.core.repository.repository.Repository.packages",
                  return_value=[package_ahriman, package_python_schedule])
-    assert repository.packages_depend_on(None) == [package_ahriman, package_python_schedule]
+    assert repository.packages_depend_on([package_ahriman, package_python_schedule], None) ==\
+        [package_ahriman, package_python_schedule]