type: update for new cors release

This commit includes following changes
* add newly added option to configuration referenec
* remove few legacy options from configuration schemas used for
  validation, which might lead to errors during validation.
  Note, however, that settings will be still read by the service
* add link to aurcache
* hide service-setup command description under spoiler

docs/configuration.rst

@@ -81,6 +81,7 @@ Base configuration settings.
 * ``apply_migrations`` - perform database migrations on the application start, boolean, optional, default ``yes``. Useful if you are using git version. Note, however, that this option must be changed only if you know what to do and going to handle migrations manually.
 * ``database`` - path to the application SQLite database, string, required.
 * ``include`` - path to directory with configuration files overrides, string, optional. Files will be read in alphabetical order.
+* ``keep_last_logs`` - amount of build logs to be kept for each package, integer, optional ,default ``0``. Logs will be cleared at the end of each process.
 * ``logging`` - path to logging configuration, string, required. Check ``logging.ini`` for reference.
 
 ``alpm:*`` groups
@@ -217,7 +218,7 @@ Mirrorlist generator plugin
 ``remote-pull`` group
 ---------------------
 
-Remote git source synchronization settings. Unlike ``Upload`` triggers those triggers are used for PKGBUILD synchronization - fetch from remote repository PKGBUILDs before updating process.
+Remote git source synchronization settings. Unlike ``upload`` triggers those triggers are used for PKGBUILD synchronization - fetch from remote repository PKGBUILDs before updating process.
 
 It supports authorization; to do so you'd need to prefix the URL with authorization part, e.g. ``https://key:token@github.com/arcan1s/ahriman.git``. It is highly recommended to use application tokens instead of your user authorization details. Alternatively, you can use any other option supported by git, e.g.:
 

docs/faq/misc.rst

@@ -56,6 +56,13 @@ Though originally I've created ahriman by trying to improve the project, it stil
 
 It is automation tools for ``repoctl`` mentioned above. Except for using shell it looks pretty cool and also offers some additional features like patches, remote synchronization (isn't it?) and reporting.
 
+`AURCache <https://github.com/Lukas-Heiligenbrunner/AURCache>`__
+""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
+
+That's really cool project if you are looking for simple service to build AUR packages. It provides very informative dashboard and easy to configure and use. However, it doesn't provide direct way to control build process (e.g. it is neither trivial to build packages for architectures which are not supported by default nor to change build flags).
+
+Also this application relies on docker setup (e.g. builders are only available as special docker containers). In addition, it uses ``paru`` to build packages instead of ``devtools``.
+
 How to check service logs
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 

docs/setup.rst

@@ -12,17 +12,20 @@ Initial setup
 
       sudo ahriman -a x86_64 -r aur service-setup ...
 
-   ``service-setup`` literally does the following steps:
+   .. admonition:: Details
+      :collapsible: closed
 
-   #.
-      Create ``/var/lib/ahriman/.makepkg.conf`` with ``makepkg.conf`` overrides if required (at least you might want to set ``PACKAGER``):
+      ``service-setup`` literally does the following steps:
 
-      .. code-block:: shell
+      #.
+         Create ``/var/lib/ahriman/.makepkg.conf`` with ``makepkg.conf`` overrides if required (at least you might want to set ``PACKAGER``):
 
-          echo 'PACKAGER="ahriman bot <ahriman@example.com>"' | sudo -u ahriman tee -a /var/lib/ahriman/.makepkg.conf
+         .. code-block:: shell
 
-   #.
-      Configure build tools (it is required for correct dependency management system):
+            echo 'PACKAGER="ahriman bot <ahriman@example.com>"' | sudo -u ahriman tee -a /var/lib/ahriman/.makepkg.conf
+
+      #.
+         Configure build tools (it is required for correct dependency management system):
 
       #.
          Create build command (you can choose any name for command, basically it should be ``{name}-{arch}-build``):
@@ -67,7 +70,7 @@ Initial setup
             echo 'ahriman ALL=(ALL) NOPASSWD:SETENV: CARCHBUILD_CMD' | tee -a /etc/sudoers.d/ahriman
             chmod 400 /etc/sudoers.d/ahriman
 
-      This command supports several arguments, kindly refer to its help message.
+   This command supports several arguments, kindly refer to its help message.
 
 #. 
    Start and enable ``ahriman@.timer`` via ``systemctl``:

package/share/ahriman/templates/build-status/package-info-modal.jinja2

@@ -60,10 +60,13 @@
                 <div class="tab-content" id="nav-tabContent">
                     <div id="package-info-logs" class="tab-pane fade show active" role="tabpanel" aria-labelledby="package-info-logs-button" tabindex="0">
                         <div class="row">
-                            <div class="col-2">
-                                <nav id="package-info-logs-versions" class="nav flex-column"></nav>
+                            <div class="col-1 dropend">
+                                <button id="package-info-logs-dropdown" class="btn dropdown-toggle" type="button" data-bs-toggle="dropdown" aria-expanded="false">
+                                    <i class="bi bi-list"></i>
+                                </button>
+                                <nav id="package-info-logs-versions" class="dropdown-menu" aria-labelledby="package-info-logs-dropdown"></nav>
                             </div>
-                            <div class="col-10">
+                            <div class="col-11">
                                 <pre class="language-console"><code id="package-info-logs-input" class="pre-scrollable language-console"></code><button id="package-info-logs-copy-button" type="button" class="btn language-console" onclick="copyLogs()"><i class="bi bi-clipboard"></i> copy</button></pre>
                             </div>
                         </div>
@@ -309,9 +312,9 @@
                     )
                     .map(version => {
                         const link = document.createElement("a");
-                        link.classList.add("nav-link");
+                        link.classList.add("dropdown-item");
 
-                        link.textContent = version.version;
+                        link.textContent = new Date(1000 * version.created).toISOStringShort();
                         link.href = "#";
                         link.onclick = _ => {
                             const logs = data

package/share/ahriman/templates/utils/style.jinja2

@@ -27,10 +27,4 @@
         top: 0;
         right: 5px;
     }
-
-    .nav-link.active {
-        pointer-events: none;
-        cursor: default;
-        color: black !important;
-    }
 </style>

src/ahriman/application/application/application.py

@@ -117,7 +117,7 @@ class Application(ApplicationPackages, ApplicationRepository):
 
         Args:
             packages(list[Package]): list of source packages of which dependencies have to be processed
-            process_dependencies(bool): if no set, dependencies will not be processed
+            process_dependencies(bool): if set to ``False``, dependencies will not be processed
 
         Returns:
             list[Package]: updated packages list. Packager for dependencies will be copied from the original package
@@ -130,6 +130,9 @@ class Application(ApplicationPackages, ApplicationRepository):
                 >>> packages = application.with_dependencies(packages, process_dependencies=True)
                 >>> application.print_updates(packages, log_fn=print)
         """
+        if not process_dependencies or not packages:
+            return packages
+
         def missing_dependencies(source: Iterable[Package]) -> dict[str, str | None]:
             # append list of known packages with packages which are in current sources
             satisfied_packages = known_packages | {
@@ -145,22 +148,29 @@ class Application(ApplicationPackages, ApplicationRepository):
                 if dependency not in satisfied_packages
             }
 
-        if not process_dependencies or not packages:
-            return packages
+        def new_packages(root: Package) -> dict[str, Package]:
+            portion = {root.base: root}
+            while missing := missing_dependencies(portion.values()):
+                for package_name, packager in missing.items():
+                    if (source_dir := self.repository.paths.cache_for(package_name)).is_dir():
+                        # there is local cache, load package from it
+                        leaf = Package.from_build(source_dir, self.repository.architecture, packager)
+                    else:
+                        leaf = Package.from_aur(package_name, packager)
+                    portion[leaf.base] = leaf
+
+                    # register package in the database
+                    self.repository.reporter.set_unknown(leaf)
+
+            return portion
 
         known_packages = self._known_packages()
-        with_dependencies = {package.base: package for package in packages}
-
-        while missing := missing_dependencies(with_dependencies.values()):
-            for package_name, username in missing.items():
-                if (source_dir := self.repository.paths.cache_for(package_name)).is_dir():
-                    # there is local cache, load package from it
-                    package = Package.from_build(source_dir, self.repository.architecture, username)
-                else:
-                    package = Package.from_aur(package_name, username)
-                with_dependencies[package.base] = package
-
-                # register package in the database
-                self.repository.reporter.set_unknown(package)
+        with_dependencies: dict[str, Package] = {}
+        for package in packages:
+            with self.in_package_context(package.base, package.version):  # use the same context for the logger
+                try:
+                    with_dependencies |= new_packages(package)
+                except Exception:
+                    self.logger.exception("could not process dependencies of %s, skip the package", package.base)
 
         return list(with_dependencies.values())

src/ahriman/application/handlers/handler.py

@@ -53,7 +53,7 @@ class Handler:
         Wrapper for all command line actions, though each derived class implements :func:`run()` method, it usually
         must not be called directly. The recommended way is to call :func:`execute()` class method, e.g.::
 
-            >>> from ahriman.application.handlers import Add
+            >>> from ahriman.application.handlers.add import Add
             >>>
             >>> Add.execute(args)
     """

src/ahriman/core/configuration/schema.py

@@ -57,10 +57,6 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
                 "path_exists": True,
                 "path_type": "file",
             },
-            "suppress_http_log_errors": {
-                "type": "boolean",
-                "coerce": "boolean",
-            }
         },
     },
     "alpm": {
@@ -347,10 +343,6 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
                 "coerce": "integer",
                 "min": 0,
             },
-            "password": {
-                "type": "string",
-                "empty": False,
-            },
             "port": {
                 "type": "integer",
                 "coerce": "integer",
@@ -379,11 +371,6 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
                 },
                 "empty": False,
             },
-            "timeout": {
-                "type": "integer",
-                "coerce": "integer",
-                "min": 0,
-            },
             "unix_socket": {
                 "type": "path",
                 "coerce": "absolute_path",
@@ -392,10 +379,6 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
                 "type": "boolean",
                 "coerce": "boolean",
             },
-            "username": {
-                "type": "string",
-                "empty": False,
-            },
             "wait_timeout": {
                 "type": "integer",
                 "coerce": "integer",

src/ahriman/core/report/report_trigger.py

@@ -67,14 +67,6 @@ class ReportTrigger(Trigger):
                     "type": "string",
                     "allowed": ["email"],
                 },
-                "full_template_path": {
-                    "type": "path",
-                    "coerce": "absolute_path",
-                    "excludes": ["template_full"],
-                    "required": True,
-                    "path_exists": True,
-                    "path_type": "file",
-                },
                 "homepage": {
                     "type": "string",
                     "empty": False,
@@ -132,26 +124,16 @@ class ReportTrigger(Trigger):
                 },
                 "template": {
                     "type": "string",
-                    "excludes": ["template_path"],
                     "dependencies": ["templates"],
                     "required": True,
                     "empty": False,
                 },
                 "template_full": {
                     "type": "string",
-                    "excludes": ["template_path"],
                     "dependencies": ["templates"],
                     "required": True,
                     "empty": False,
                 },
-                "template_path": {
-                    "type": "path",
-                    "coerce": "absolute_path",
-                    "excludes": ["template"],
-                    "required": True,
-                    "path_exists": True,
-                    "path_type": "file",
-                },
                 "templates": {
                     "type": "list",
                     "coerce": "list",
@@ -199,19 +181,10 @@ class ReportTrigger(Trigger):
                 },
                 "template": {
                     "type": "string",
-                    "excludes": ["template_path"],
                     "dependencies": ["templates"],
                     "required": True,
                     "empty": False,
                 },
-                "template_path": {
-                    "type": "path",
-                    "coerce": "absolute_path",
-                    "excludes": ["template"],
-                    "required": True,
-                    "path_exists": True,
-                    "path_type": "file",
-                },
                 "templates": {
                     "type": "list",
                     "coerce": "list",
@@ -225,76 +198,6 @@ class ReportTrigger(Trigger):
                 },
             },
         },
-        "telegram": {
-            "type": "dict",
-            "schema": {
-                "type": {
-                    "type": "string",
-                    "allowed": ["telegram"],
-                },
-                "api_key": {
-                    "type": "string",
-                    "required": True,
-                    "empty": False,
-                },
-                "chat_id": {
-                    "type": "string",
-                    "required": True,
-                    "empty": False,
-                },
-                "homepage": {
-                    "type": "string",
-                    "empty": False,
-                    "is_url": ["http", "https"],
-                },
-                "link_path": {
-                    "type": "string",
-                    "required": True,
-                    "empty": False,
-                    "is_url": [],
-                },
-                "rss_url": {
-                    "type": "string",
-                    "empty": False,
-                    "is_url": ["http", "https"],
-                },
-                "template": {
-                    "type": "string",
-                    "excludes": ["template_path"],
-                    "dependencies": ["templates"],
-                    "required": True,
-                    "empty": False,
-                },
-                "template_path": {
-                    "type": "path",
-                    "coerce": "absolute_path",
-                    "excludes": ["template"],
-                    "required": True,
-                    "path_exists": True,
-                    "path_type": "file",
-                },
-                "template_type": {
-                    "type": "string",
-                    "allowed": ["MarkdownV2", "HTML", "Markdown"],
-                },
-                "templates": {
-                    "type": "list",
-                    "coerce": "list",
-                    "schema": {
-                        "type": "path",
-                        "coerce": "absolute_path",
-                        "path_exists": True,
-                        "path_type": "dir",
-                    },
-                    "empty": False,
-                },
-                "timeout": {
-                    "type": "integer",
-                    "coerce": "integer",
-                    "min": 0,
-                },
-            },
-        },
         "remote-call": {
             "type": "dict",
             "schema": {
@@ -354,19 +257,10 @@ class ReportTrigger(Trigger):
                 },
                 "template": {
                     "type": "string",
-                    "excludes": ["template_path"],
                     "dependencies": ["templates"],
                     "required": True,
                     "empty": False,
                 },
-                "template_path": {
-                    "type": "path",
-                    "coerce": "absolute_path",
-                    "excludes": ["template"],
-                    "required": True,
-                    "path_exists": True,
-                    "path_type": "file",
-                },
                 "templates": {
                     "type": "list",
                     "coerce": "list",
@@ -380,6 +274,67 @@ class ReportTrigger(Trigger):
                 },
             },
         },
+        "telegram": {
+            "type": "dict",
+            "schema": {
+                "type": {
+                    "type": "string",
+                    "allowed": ["telegram"],
+                },
+                "api_key": {
+                    "type": "string",
+                    "required": True,
+                    "empty": False,
+                },
+                "chat_id": {
+                    "type": "string",
+                    "required": True,
+                    "empty": False,
+                },
+                "homepage": {
+                    "type": "string",
+                    "empty": False,
+                    "is_url": ["http", "https"],
+                },
+                "link_path": {
+                    "type": "string",
+                    "required": True,
+                    "empty": False,
+                    "is_url": [],
+                },
+                "rss_url": {
+                    "type": "string",
+                    "empty": False,
+                    "is_url": ["http", "https"],
+                },
+                "template": {
+                    "type": "string",
+                    "dependencies": ["templates"],
+                    "required": True,
+                    "empty": False,
+                },
+                "template_type": {
+                    "type": "string",
+                    "allowed": ["MarkdownV2", "HTML", "Markdown"],
+                },
+                "templates": {
+                    "type": "list",
+                    "coerce": "list",
+                    "schema": {
+                        "type": "path",
+                        "coerce": "absolute_path",
+                        "path_exists": True,
+                        "path_type": "dir",
+                    },
+                    "empty": False,
+                },
+                "timeout": {
+                    "type": "integer",
+                    "coerce": "integer",
+                    "min": 0,
+                },
+            },
+        },
     }
 
     def __init__(self, repository_id: RepositoryId, configuration: Configuration) -> None:

src/ahriman/core/upload/upload_trigger.py

@@ -83,6 +83,20 @@ class UploadTrigger(Trigger):
                 },
             },
         },
+        "remote-service": {
+            "type": "dict",
+            "schema": {
+                "type": {
+                    "type": "string",
+                    "allowed": ["ahriman", "remote-service"],
+                },
+                "timeout": {
+                    "type": "integer",
+                    "coerce": "integer",
+                    "min": 0,
+                },
+            },
+        },
         "rsync": {
             "type": "dict",
             "schema": {
@@ -107,20 +121,6 @@ class UploadTrigger(Trigger):
                 },
             },
         },
-        "remote-service": {
-            "type": "dict",
-            "schema": {
-                "type": {
-                    "type": "string",
-                    "allowed": ["ahriman", "remote-service"],
-                },
-                "timeout": {
-                    "type": "integer",
-                    "coerce": "integer",
-                    "min": 0,
-                },
-            },
-        },
         "s3": {
             "type": "dict",
             "schema": {

src/ahriman/web/cors.py

@@ -17,7 +17,7 @@
 # You should have received a copy of the GNU General Public License
 # along with this program. If not, see <http://www.gnu.org/licenses/>.
 #
-import aiohttp_cors  # type: ignore[import-untyped]
+import aiohttp_cors
 
 from aiohttp.web import Application
 
@@ -36,7 +36,7 @@ def setup_cors(application: Application) -> aiohttp_cors.CorsConfig:
         aiohttp_cors.CorsConfig: generated CORS configuration
     """
     cors = aiohttp_cors.setup(application, defaults={
-        "*": aiohttp_cors.ResourceOptions(
+        "*": aiohttp_cors.ResourceOptions(  # type: ignore[no-untyped-call]
             expose_headers="*",
             allow_headers="*",
             allow_methods="*",

src/ahriman/web/views/base.py

@@ -18,7 +18,7 @@
 # along with this program. If not, see <http://www.gnu.org/licenses/>.
 #
 from aiohttp.web import HTTPBadRequest, HTTPNotFound, Request, StreamResponse, View
-from aiohttp_cors import CorsViewMixin  # type: ignore[import-untyped]
+from aiohttp_cors import CorsViewMixin
 from collections.abc import Awaitable, Callable
 from typing import ClassVar, TypeVar
 

tests/ahriman/application/application/test_application.py

@@ -113,6 +113,40 @@ def test_with_dependencies(application: Application, package_ahriman: Package, p
     ], any_order=True)
 
 
+def test_with_dependencies_exception(application: Application, package_ahriman: Package,
+                                     package_python_schedule: Package, mocker: MockerFixture) -> None:
+    """
+    must skip packages if exception occurs
+    """
+    def create_package_mock(package_base) -> MagicMock:
+        mock = MagicMock()
+        mock.base = package_base
+        mock.depends_build = []
+        mock.packages_full = [package_base]
+        return mock
+
+    package_python_schedule.packages = {
+        package_python_schedule.base: package_python_schedule.packages[package_python_schedule.base]
+    }
+    package_ahriman.packages[package_ahriman.base].depends = ["devtools", "python", package_python_schedule.base]
+    package_ahriman.packages[package_ahriman.base].make_depends = ["python-build", "python-installer"]
+
+    packages = {
+        package_ahriman.base: package_ahriman,
+        package_python_schedule.base: package_python_schedule,
+        "python": create_package_mock("python"),
+        "python-installer": create_package_mock("python-installer"),
+    }
+
+    mocker.patch("pathlib.Path.is_dir", autospec=True, side_effect=lambda p: p.name == "python")
+    mocker.patch("ahriman.models.package.Package.from_aur", side_effect=lambda *args: packages[args[0]])
+    mocker.patch("ahriman.models.package.Package.from_build", side_effect=Exception)
+    mocker.patch("ahriman.application.application.Application._known_packages",
+                 return_value={"devtools", "python-build", "python-pytest"})
+
+    assert not application.with_dependencies([package_ahriman], process_dependencies=True)
+
+
 def test_with_dependencies_skip(application: Application, package_ahriman: Package, mocker: MockerFixture) -> None:
     """
     must skip processing of dependencies