raise 404 in case if repository is unknown

The old solution didn't actually work as expected, because devtools
configuration belongs to filesystem (as well as sudo one), so it was
still required to run setup command.

In order to handle additional repositories, the POSTSETUP and PRESETUP
commands variables have been introduced. FAQ has been updated as well

CONTRIBUTING.md

@@ -202,6 +202,7 @@ Again, the most checks can be performed by `make check` command, though some add
                 400: {"description": "Bad data is supplied", "schema": ErrorSchema},  # exception raised by this method
                 401: {"description": "Authorization required", "schema": ErrorSchema},  # should be always presented
                 403: {"description": "Access is forbidden", "schema": ErrorSchema},  # should be always presented
+                404: {"description": "Repository is unknown", "schema": ErrorSchema},  # include if BaseView.service() method is called
                 500: {"description": "Internal server error", "schema": ErrorSchema},  # should be always presented
             },
             security=[{"token": [POST_PERMISSION]}],

Dockerfile

@@ -10,6 +10,8 @@ ENV AHRIMAN_OUTPUT=""
 ENV AHRIMAN_PACKAGER="ahriman bot <ahriman@example.com>"
 ENV AHRIMAN_PACMAN_MIRROR=""
 ENV AHRIMAN_PORT=""
+ENV AHRIMAN_POSTSETUP_COMMAND=""
+ENV AHRIMAN_PRESETUP_COMMAND=""
 ENV AHRIMAN_REPOSITORY="aur-clone"
 ENV AHRIMAN_REPOSITORY_SERVER=""
 ENV AHRIMAN_REPOSITORY_ROOT="/var/lib/ahriman/ahriman"

docker/entrypoint.sh

@@ -49,7 +49,10 @@ fi
 if [ -n "$AHRIMAN_UNIX_SOCKET" ]; then
     AHRIMAN_SETUP_ARGS+=("--web-unix-socket" "$AHRIMAN_UNIX_SOCKET")
 fi
+
+[ -n "$AHRIMAN_PRESETUP_COMMAND" ] && eval "$AHRIMAN_PRESETUP_COMMAND"
 ahriman "${AHRIMAN_DEFAULT_ARGS[@]}" service-setup "${AHRIMAN_SETUP_ARGS[@]}"
+[ -n "$AHRIMAN_POSTSETUP_COMMAND" ] && eval "$AHRIMAN_POSTSETUP_COMMAND"
 
 # validate configuration if set
 [ -n "$AHRIMAN_VALIDATE_CONFIGURATION" ] && ahriman "${AHRIMAN_DEFAULT_ARGS[@]}" service-config-validate --exit-code

docs/configuration.rst

@@ -41,7 +41,7 @@ Base configuration settings.
 
 * ``apply_migrations`` - perform migrations on application start, boolean, optional, default ``yes``. Useful if you are using git version. Note, however, that this option must be changed only if you know what to do and going to handle migrations automatically.
 * ``database`` - path to SQLite database, string, required.
-* ``include`` - path to directory with configuration files overrides, string, optional. Note, however, that the application will also load configuration files from the repository root, which is used, in particular, by setup subcommand.
+* ``include`` - path to directory with configuration files overrides, string, optional.
 * ``logging`` - path to logging configuration, string, required. Check ``logging.ini`` for reference.
 * ``suppress_http_log_errors`` - suppress http log errors, boolean, optional, default ``no``. If set to ``yes``, any http log errors (e.g. if web server is not available, but http logging is enabled) will be suppressed.
 

docs/faq.rst

@@ -402,6 +402,8 @@ The following environment variables are supported:
 * ``AHRIMAN_PACKAGER`` - packager name from which packages will be built, default is ``ahriman bot <ahriman@example.com>``.
 * ``AHRIMAN_PACMAN_MIRROR`` - override pacman mirror server if set.
 * ``AHRIMAN_PORT`` - HTTP server port if any, default is empty.
+* ``AHRIMAN_POSTSETUP_COMMAND`` - if set, the command which will be called (as root) after the setup command, but before any other actions.
+* ``AHRIMAN_PRESETUP_COMMAND`` - if set, the command which will be called (as root) right before the setup command.
 * ``AHRIMAN_REPOSITORY`` - repository name, default is ``aur-clone``.
 * ``AHRIMAN_REPOSITORY_SERVER`` - optional override for the repository url. Useful if you would like to download packages from remote instead of local filesystem.
 * ``AHRIMAN_REPOSITORY_ROOT`` - repository root. Because of filesystem rights it is required to override default repository root. By default, it uses ``ahriman`` directory inside ahriman's home, which can be passed as mount volume.
@@ -429,7 +431,7 @@ This command uses same rules as ``repo-update``, thus, e.g. requires ``--privile
 Web service setup
 ^^^^^^^^^^^^^^^^^
 
-Well for that you would need to have web container instance running forever; it can be achieved by the following command:
+For that you would need to have web container instance running forever; it can be achieved by the following command:
 
 .. code-block:: shell
 
@@ -451,6 +453,20 @@ Otherwise, you would need to pass ``AHRIMAN_PORT`` and mount container network t
 
    docker run --privileged --net=host -e AHRIMAN_PORT=8080 -v /path/to/local/repo:/var/lib/ahriman arcan1s/ahriman:latest
 
+Mutli-repository web service
+""""""""""""""""""""""""""""
+
+Idea is pretty same as to just run web service. However, it is required to run setup commands for each repository, except for one which is specified by ``AHRIMAN_REPOSITORY`` and ``AHRIMAN_ARCHITECTURE ` variables.
+
+In order to create configuration for additional repositories, the ``AHRIMAN_POSTSETUP_COMMAND`` variable should be used, e.g.:
+
+.. code-block:: shell
+
+   docker run --privileged -p 8080:8080 -e AHRIMAN_PORT=8080 -e AHRIMAN_UNIX_SOCKET=/var/lib/ahriman/ahriman/ahriman-web.sock -e AHRIMAN_POSTSETUP_COMMAND="ahriman --architecture x86_64 --repository aur-clone-v2 service-setup --build-as-user ahriman --packager 'ahriman bot <ahriman@example.com>'" -v /path/to/local/repo:/var/lib/ahriman arcan1s/ahriman:latest
+
+The command above will also create configuration for the repository named ``aur-clone-v2``.
+
+Note, however, that the command above is only required in case if the service is going to be used to run subprocesses. Otherwise, everything else (web interface, status, etc) will be handled as usual.
 
 Non-x86_64 architecture setup
 -----------------------------

package/share/ahriman/templates/build-status.jinja2

@@ -17,7 +17,7 @@
 
         <div class="container">
             <nav class="navbar navbar-expand-lg">
-                <a class="navbar-brand" href="#"><img src="/static/logo.svg" width="30" height="30" alt=""> ahriman</a>
+                <div class="navbar-brand"><img src="/static/logo.svg" width="30" height="30" alt=""> ahriman</div>
                 <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbar-supported-content" aria-controls="navbar-supported-content" aria-expanded="false" aria-label="Toggle navigation">
                     <span class="navbar-toggler-icon"></span>
                 </button>

src/ahriman/application/handlers/setup.py

@@ -135,7 +135,7 @@ class Setup(Handler):
             configuration.set_option("auth", "salt", User.generate_password(20))
 
         (root.include / "00-setup-overrides.ini").unlink(missing_ok=True)  # remove old-style configuration
-        target = root.repository_paths.root / f"00-setup-overrides-{repository_id.id}.ini"
+        target = root.include / f"00-setup-overrides-{repository_id.id}.ini"
         with target.open("w") as ahriman_configuration:
             configuration.write(ahriman_configuration)
 

src/ahriman/application/lock.py

@@ -70,9 +70,8 @@ class Lock(LazyLogging):
             repository_id(RepositoryId): repository unique identifier
             configuration(Configuration): configuration instance
         """
-        lock_suffix = f"{repository_id.name}_{repository_id.architecture}"
         self.path: Path | None = \
-            args.lock.with_stem(f"{args.lock.stem}_{lock_suffix}") if args.lock is not None else None
+            args.lock.with_stem(f"{args.lock.stem}_{repository_id.id}") if args.lock is not None else None
 
         self.force: bool = args.force
         self.unsafe: bool = args.unsafe

src/ahriman/core/configuration/configuration.py

@@ -284,18 +284,18 @@ class Configuration(configparser.RawConfigParser):
         self.path = path
 
         self.read(self.path)
-
-        self.includes = []  # reset state
-        self.load_includes()  # basic includes
-        self.load_includes(self.getpath("repository", "root"))  # home directory includes
+        self.load_includes()  # load includes
 
     def load_includes(self, path: Path | None = None) -> None:
         """
         load configuration includes from specified path
 
         Args:
-            path(Path | None, optional): path to directory with include files (Default value = None)
+            path(Path | None, optional): path to directory with include files. If none set, the default path will be
+                used (Default value = None)
         """
+        self.includes = []  # reset state
+
         try:
             path = path or self.include
             for include in sorted(path.glob("*.ini")):

src/ahriman/web/views/base.py

@@ -18,7 +18,7 @@
 # along with this program. If not, see <http://www.gnu.org/licenses/>.
 #
 from aiohttp_cors import CorsViewMixin  # type: ignore[import-untyped]
-from aiohttp.web import HTTPBadRequest, Request, StreamResponse, View
+from aiohttp.web import HTTPBadRequest, HTTPNotFound, Request, StreamResponse, View
 from collections.abc import Awaitable, Callable
 from typing import Any, TypeVar
 
@@ -245,10 +245,16 @@ class BaseView(View, CorsViewMixin):
 
         Returns:
             Watcher: build status watcher instance. If no repository provided, it will return the first one
+
+        Raises:
+            HTTPNotFound: if no repository found
         """
         if repository_id is None:
             repository_id = self.repository_id()
-        return self.services[repository_id]
+        try:
+            return self.services[repository_id]
+        except KeyError:
+            raise HTTPNotFound(reason=f"Repository {repository_id.id} is unknown")
 
     async def username(self) -> str | None:
         """

src/ahriman/web/views/v1/service/add.py

@@ -46,6 +46,7 @@ class AddView(BaseView):
             400: {"description": "Bad data is supplied", "schema": ErrorSchema},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
+            404: {"description": "Repository is unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [POST_PERMISSION]}],

src/ahriman/web/views/v1/service/pgp.py

@@ -48,7 +48,7 @@ class PGPView(BaseView):
             400: {"description": "Bad data is supplied", "schema": ErrorSchema},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
-            404: {"description": "Package base is unknown", "schema": ErrorSchema},
+            404: {"description": "PGP key is unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [GET_PERMISSION]}],
@@ -75,7 +75,7 @@ class PGPView(BaseView):
         try:
             key = self.sign.key_download(server, key)
         except Exception:
-            raise HTTPNotFound
+            raise HTTPNotFound(reason=f"Key {key} is unknown")
 
         return json_response({"key": key})
 

src/ahriman/web/views/v1/service/process.py

@@ -45,7 +45,7 @@ class ProcessView(BaseView):
             200: {"description": "Success response", "schema": ProcessSchema},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
-            404: {"description": "Not found", "schema": ErrorSchema},
+            404: {"description": "Process is unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [GET_PERMISSION]}],

src/ahriman/web/views/v1/service/rebuild.py

@@ -46,6 +46,7 @@ class RebuildView(BaseView):
             400: {"description": "Bad data is supplied", "schema": ErrorSchema},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
+            404: {"description": "Repository is unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [POST_PERMISSION]}],

src/ahriman/web/views/v1/service/remove.py

@@ -46,6 +46,7 @@ class RemoveView(BaseView):
             400: {"description": "Bad data is supplied", "schema": ErrorSchema},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
+            404: {"description": "Repository is unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [POST_PERMISSION]}],

src/ahriman/web/views/v1/service/request.py

@@ -46,6 +46,7 @@ class RequestView(BaseView):
             400: {"description": "Bad data is supplied", "schema": ErrorSchema},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
+            404: {"description": "Repository is unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [POST_PERMISSION]}],

src/ahriman/web/views/v1/service/update.py

@@ -46,6 +46,7 @@ class UpdateView(BaseView):
             400: {"description": "Bad data is supplied", "schema": ErrorSchema},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
+            404: {"description": "Repository is unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [POST_PERMISSION]}],

src/ahriman/web/views/v1/service/upload.py

@@ -101,7 +101,7 @@ class UploadView(BaseView):
             400: {"description": "Bad data is supplied", "schema": ErrorSchema},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
-            404: {"description": "Not found", "schema": ErrorSchema},
+            404: {"description": "Repository is unknown or endpoint is disabled", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [POST_PERMISSION]}],

src/ahriman/web/views/v1/status/logs.py

@@ -51,6 +51,7 @@ class LogsView(BaseView):
             204: {"description": "Success response"},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
+            404: {"description": "Repository is unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [DELETE_PERMISSION]}],
@@ -78,7 +79,7 @@ class LogsView(BaseView):
             200: {"description": "Success response", "schema": LogsSchema},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
-            404: {"description": "Package base is unknown", "schema": ErrorSchema},
+            404: {"description": "Package base and/or repository are unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [GET_PERMISSION]}],
@@ -101,7 +102,7 @@ class LogsView(BaseView):
         try:
             _, status = self.service().package_get(package_base)
         except UnknownPackageError:
-            raise HTTPNotFound
+            raise HTTPNotFound(reason=f"Package {package_base} is unknown")
         logs = self.service().logs_get(package_base)
 
         response = {
@@ -120,6 +121,7 @@ class LogsView(BaseView):
             400: {"description": "Bad data is supplied", "schema": ErrorSchema},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
+            404: {"description": "Repository is unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [POST_PERMISSION]}],

src/ahriman/web/views/v1/status/package.py

@@ -52,6 +52,7 @@ class PackageView(BaseView):
             204: {"description": "Success response"},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
+            404: {"description": "Repository is unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [DELETE_PERMISSION]}],
@@ -79,7 +80,7 @@ class PackageView(BaseView):
             200: {"description": "Success response", "schema": PackageStatusSchema(many=True)},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
-            404: {"description": "Package base is unknown", "schema": ErrorSchema},
+            404: {"description": "Package base and/or repository are unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [GET_PERMISSION]}],
@@ -103,7 +104,7 @@ class PackageView(BaseView):
         try:
             package, status = self.service(repository_id).package_get(package_base)
         except UnknownPackageError:
-            raise HTTPNotFound
+            raise HTTPNotFound(reason=f"Package {package_base} is unknown")
 
         response = [
             {
@@ -123,6 +124,7 @@ class PackageView(BaseView):
             400: {"description": "Bad data is supplied", "schema": ErrorSchema},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
+            404: {"description": "Repository is unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [POST_PERMISSION]}],

src/ahriman/web/views/v1/status/packages.py

@@ -52,6 +52,7 @@ class PackagesView(BaseView):
             400: {"description": "Bad data is supplied", "schema": ErrorSchema},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
+            404: {"description": "Repository is unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [GET_PERMISSION]}],
@@ -90,6 +91,7 @@ class PackagesView(BaseView):
             204: {"description": "Success response"},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
+            404: {"description": "Repository is unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [POST_PERMISSION]}],

src/ahriman/web/views/v1/status/status.py

@@ -51,6 +51,7 @@ class StatusView(BaseView):
             200: {"description": "Success response", "schema": InternalStatusSchema},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
+            404: {"description": "Repository is unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [GET_PERMISSION]}],
@@ -84,6 +85,7 @@ class StatusView(BaseView):
             400: {"description": "Bad data is supplied", "schema": ErrorSchema},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
+            404: {"description": "Repository is unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [POST_PERMISSION]}],

src/ahriman/web/views/v2/status/logs.py

@@ -47,7 +47,7 @@ class LogsView(BaseView):
             400: {"description": "Bad data is supplied", "schema": ErrorSchema},
             401: {"description": "Authorization required", "schema": ErrorSchema},
             403: {"description": "Access is forbidden", "schema": ErrorSchema},
-            404: {"description": "Package base is unknown", "schema": ErrorSchema},
+            404: {"description": "Package base and/or repository are unknown", "schema": ErrorSchema},
             500: {"description": "Internal server error", "schema": ErrorSchema},
         },
         security=[{"token": [GET_PERMISSION]}],
@@ -71,7 +71,7 @@ class LogsView(BaseView):
         try:
             _, status = self.service().package_get(package_base)
         except UnknownPackageError:
-            raise HTTPNotFound
+            raise HTTPNotFound(reason=f"Package {package_base} is unknown")
         logs = self.service().logs_get(package_base, limit, offset)
 
         response = {

tests/ahriman/application/test_lock.py

@@ -23,7 +23,7 @@ def test_path(args: argparse.Namespace, configuration: Configuration) -> None:
     assert Lock(args, repository_id, configuration).path is None
 
     args.lock = Path("/run/ahriman.lock")
-    assert Lock(args, repository_id, configuration).path == Path("/run/ahriman_aur-clone_x86_64.lock")
+    assert Lock(args, repository_id, configuration).path == Path("/run/ahriman_x86_64-aur-clone.lock")
 
     with pytest.raises(ValueError):
         args.lock = Path("/")

tests/ahriman/conftest.py

@@ -245,19 +245,17 @@ def auth(configuration: Configuration) -> Auth:
 
 
 @pytest.fixture
-def configuration(repository_id: RepositoryId, resource_path_root: Path, mocker: MockerFixture) -> Configuration:
+def configuration(repository_id: RepositoryId, resource_path_root: Path) -> Configuration:
     """
     configuration fixture
 
     Args:
         repository_id(RepositoryId): repository identifier fixture
         resource_path_root(Path): resource path root directory
-        mocker(MockerFixture): mocker object
 
     Returns:
         Configuration: configuration test instance
     """
-    mocker.patch("ahriman.core.configuration.Configuration.load_includes")
     path = resource_path_root / "core" / "ahriman.ini"
     return Configuration.from_path(path, repository_id)
 

tests/ahriman/core/configuration/test_configuration.py

@@ -45,10 +45,7 @@ def test_from_path(repository_id: RepositoryId, mocker: MockerFixture) -> None:
     configuration = Configuration.from_path(path, repository_id)
     assert configuration.path == path
     read_mock.assert_called_once_with(path)
-    load_includes_mock.assert_has_calls([
-        MockCall(),
-        MockCall(configuration.repository_paths.root),
-    ])
+    load_includes_mock.assert_called_once_with()
 
 
 def test_from_path_file_missing(repository_id: RepositoryId, mocker: MockerFixture) -> None:

tests/ahriman/web/views/test_view_base.py

@@ -2,7 +2,7 @@ import pytest
 
 from multidict import MultiDict
 from aiohttp.test_utils import TestClient
-from aiohttp.web import HTTPBadRequest
+from aiohttp.web import HTTPBadRequest, HTTPNotFound
 from pytest_mock import MockerFixture
 from unittest.mock import AsyncMock
 
@@ -240,6 +240,14 @@ def test_service_auto(base: BaseView, repository_id: RepositoryId, mocker: Mocke
     assert base.service() == base.services[repository_id]
 
 
+def test_service_not_found(base: BaseView) -> None:
+    """
+    must raise HTTPNotFound if no repository found
+    """
+    with pytest.raises(HTTPNotFound):
+        base.service(RepositoryId("", ""))
+
+
 async def test_username(base: BaseView, mocker: MockerFixture) -> None:
     """
     must return identity of logged-in user