arcanis/ahriman
1
0
Fork 0
mirror of https://github.com/arcan1s/ahriman.git synced 2025-04-24 15:27:17 +00:00
Code Issues Projects Releases Wiki Activity

docs update, config validation rules update

Browse Source
This commit is contained in:
Evgenii Alekseev 2023-09-08 23:08:11 +03:00
parent a02baec662
commit 9823dc9061
18 changed files with 217 additions and 30 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
docs/configuration.rst
View File

@ -118,7 +118,7 @@ Web server settings. If any of ``host``/``port`` is not set, web integration wil
* ``password`` - password to authorize in web service in order to update service status, string, required in case if authorization enabled.
* ``port`` - port to bind, int, optional.
* ``static_path`` - path to directory with static files, string, required.
* ``templates`` - path to templates directory, string, required.
* ``templates`` - path to templates directories, space separated list of strings, required.
* ``timeout`` - HTTP request timeout in seconds, int, optional, default is ``30``.
* ``unix_socket`` - path to the listening unix socket, string, optional. If set, server will create the socket on the specified address which can (and will) be used by application. Note, that unlike usual host/port configuration, unix socket allows to perform requests without authorization.
* ``unix_socket_unsafe`` - set unsafe (o+w) permissions to unix socket, boolean, optional, default ``yes``. This option is enabled by default, because it is supposed that unix socket is created in safe environment (only web service is supposed to be used in unsafe), but it can be disabled by configuration.
@ -231,7 +231,6 @@ Section name must be either ``console`` (plus optional architecture name, e.g. `
Section name must be either ``email`` (plus optional architecture name, e.g. ``email:x86_64``) or random name with ``type`` set.
* ``type`` - type of the report, string, optional, must be set to ``email`` if exists.
* ``full_template_path`` - path to Jinja2 template for full package description index, string, optional.
* ``homepage`` - link to homepage, string, optional.
* ``host`` - SMTP host for sending emails, string, required.
* ``link_path`` - prefix for HTML links, string, required.
@ -241,7 +240,9 @@ Section name must be either ``email`` (plus optional architecture name, e.g. ``e
* ``receivers`` - SMTP receiver addresses, space separated list of strings, required.
* ``sender`` - SMTP sender address, string, required.
* ``ssl`` - SSL mode for SMTP connection, one of ``ssl``, ``starttls``, ``disabled``, optional, default ``disabled``.
* ``template_path`` - path to Jinja2 template, string, required.
* ``template`` - Jinja2 template name, string, required.
* ``template_full`` - Jinja2 template name for full package description index, string, optional.
* ``templates`` - path to templates directories, space separated list of strings, required.
* ``user`` - SMTP user to authenticate, string, optional.
``html`` type
@ -253,7 +254,8 @@ Section name must be either ``html`` (plus optional architecture name, e.g. ``ht
* ``homepage`` - link to homepage, string, optional.
* ``link_path`` - prefix for HTML links, string, required.
* ``path`` - path to html report file, string, required.
* ``template_path`` - path to Jinja2 template, string, required.
* ``template`` - Jinja2 template name, string, required.
* ``templates`` - path to templates directories, space separated list of strings, required.
``remote-call`` type
^^^^^^^^^^^^^^^^^^^^
@ -276,8 +278,9 @@ Section name must be either ``telegram`` (plus optional architecture name, e.g.
* ``chat_id`` - telegram chat id, either string with ``@`` or integer value, required.
* ``homepage`` - link to homepage, string, optional.
* ``link_path`` - prefix for HTML links, string, required.
* ``template_path`` - path to Jinja2 template, string, required.
* ``template`` - Jinja2 template name, string, required.
* ``template_type`` - ``parse_mode`` to be passed to telegram API, one of ``MarkdownV2``, ``HTML``, ``Markdown``, string, optional, default ``HTML``.
* ``templates`` - path to templates directories, space separated list of strings, required.
* ``timeout`` - HTTP request timeout in seconds, int, optional, default is ``30``.
``upload`` group

4
docs/faq.rst
View File

@ -1299,7 +1299,9 @@ The application uses java concept to log messages, e.g. class ``Application`` im
Html customization
^^^^^^^^^^^^^^^^^^
It is possible to customize html templates. In order to do so, create files somewhere (refer to Jinja2 documentation and the service source code for available parameters) and put ``template_path`` to configuration pointing to this directory.
It is possible to customize html templates. In order to do so, create files somewhere (refer to Jinja2 documentation and the service source code for available parameters) and prepend ``templates`` with value pointing to this directory.
In addition, default html templates supports style customization out-of-box. In order to customize style, just put file named ``user-style.jinja2`` to the templates directory.
I did not find my question
^^^^^^^^^^^^^^^^^^^^^^^^^^

1
package/share/ahriman/templates/build-status.jinja2
View File

@ -8,6 +8,7 @@
<link rel="shortcut icon" href="/static/favicon.ico">
{% include "utils/style.jinja2" %}
{% include "user-style.jinja2" ignore missing %}
</head>
<body>

1
package/share/ahriman/templates/email-index.jinja2
View File

@ -7,6 +7,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1">
{% include "utils/style.jinja2" %}
{% include "user-style.jinja2" ignore missing %}
</head>
<body>

1
package/share/ahriman/templates/error.jinja2
View File

@ -8,6 +8,7 @@
<link rel="shortcut icon" href="/static/favicon.ico">
{% include "utils/style.jinja2" %}
{% include "user-style.jinja2" ignore missing %}
</head>
<body>

1
package/share/ahriman/templates/repo-index.jinja2
View File

@ -6,6 +6,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1">
{% include "utils/style.jinja2" %}
{% include "user-style.jinja2" ignore missing %}
</head>
<body>

63
src/ahriman/core/configuration/schema.py
View File

@ -44,12 +44,14 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"coerce": "absolute_path",
"required": True,
"path_exists": True,
"path_type": "dir",
},
"logging": {
"type": "path",
"coerce": "absolute_path",
"required": True,
"path_exists": True,
"path_type": "file",
},
"suppress_http_log_errors": {
"type": "boolean",
@ -68,12 +70,16 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"mirror": {
"type": "string",
"required": True,
"empty": False,
"is_url": [],
},
"repositories": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
"schema": {
"type": "string",
"empty": False,
},
"required": True,
"empty": False,
},
@ -82,6 +88,7 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"coerce": "absolute_path",
"required": True,
"path_exists": True,
"path_type": "dir",
},
"use_ahriman_cache": {
"type": "boolean",
@ -113,9 +120,11 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
},
"client_id": {
"type": "string",
"empty": False,
},
"client_secret": {
"type": "string",
"empty": False,
},
"cookie_secret_key": {
"type": "string",
@ -129,9 +138,11 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
},
"oauth_provider": {
"type": "string",
"empty": False,
},
"oauth_scopes": {
"type": "string",
"empty": False,
},
"salt": {
"type": "string",
@ -144,36 +155,55 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"archbuild_flags": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
"schema": {
"type": "string",
"empty": False,
},
},
"build_command": {
"type": "string",
"required": True,
"empty": False,
},
"ignore_packages": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
"schema": {
"type": "string",
"empty": False,
},
},
"makepkg_flags": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
"schema": {
"type": "string",
"empty": False,
},
},
"makechrootpkg_flags": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
"schema": {
"type": "string",
"empty": False,
},
},
"triggers": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
"schema": {
"type": "string",
"empty": False,
},
},
"triggers_known": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
"schema": {
"type": "string",
"empty": False,
},
},
"vcs_allowed_age": {
"type": "integer",
@ -187,10 +217,14 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"schema": {
"name": {
"type": "string",
"empty": False,
},
"root": {
"type": "string",
"type": "path",
"coerce": "absolute_path",
"required": True,
"path_exists": True,
"path_type": "dir",
},
},
},
@ -208,6 +242,7 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
},
"key": {
"type": "string",
"empty": False,
},
},
},
@ -216,6 +251,7 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"schema": {
"address": {
"type": "string",
"empty": False,
"is_url": ["http", "https"],
},
"debug": {
@ -229,7 +265,10 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"debug_allowed_hosts": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
"schema": {
"type": "string",
"empty": False,
},
},
"enable_archive_upload": {
"type": "boolean",
@ -237,10 +276,12 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
},
"host": {
"type": "string",
"empty": False,
"is_ip_address": ["localhost"],
},
"index_url": {
"type": "string",
"empty": False,
"is_url": ["http", "https"],
},
"max_body_size": {
@ -250,6 +291,7 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
},
"password": {
"type": "string",
"empty": False,
},
"port": {
"type": "integer",
@ -262,6 +304,7 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"coerce": "absolute_path",
"required": True,
"path_exists": True,
"path_type": "dir",
},
"templates": {
"type": "list",
@ -270,6 +313,7 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"type": "path",
"coerce": "absolute_path",
"path_exists": True,
"path_type": "dir",
},
"empty": False,
},
@ -288,6 +332,7 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
},
"username": {
"type": "string",
"empty": False,
},
"wait_timeout": {
"type": "integer",

18
src/ahriman/core/configuration/validator.py
View File

@ -162,3 +162,21 @@ class Validator(RootValidator):
self._error(field, f"Path {value} must not exist")
case False if constraint:
self._error(field, f"Path {value} must exist")
def _validate_path_type(self, constraint: str, field: str, value: Path) -> None:
"""
check if paths is file, directory or whatever. The match will be performed as call of ``is_{constraint}``
method of the path object
Args:
constraint(str): path type to be matched
field(str): field name to be checked
value(Path): value to be checked
Examples:
The rule's arguments are validated against this schema:
{"type": "string"}
"""
fn = getattr(value, f"is_{constraint}")
if not fn():
self._error(field, f"Path {value} must be type of {constraint}")

7
src/ahriman/core/gitremote/remote_pull_trigger.py
View File

@ -38,7 +38,10 @@ class RemotePullTrigger(Trigger):
"target": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
"schema": {
"type": "string",
"empty": False,
},
},
},
},
@ -48,9 +51,11 @@ class RemotePullTrigger(Trigger):
"pull_url": {
"type": "string",
"required": True,
"empty": False,
},
"pull_branch": {
"type": "string",
"empty": False,
},
},
},

9
src/ahriman/core/gitremote/remote_push_trigger.py
View File

@ -43,7 +43,10 @@ class RemotePushTrigger(Trigger):
"target": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
"schema": {
"type": "string",
"empty": False,
},
},
},
},
@ -52,16 +55,20 @@ class RemotePushTrigger(Trigger):
"schema": {
"commit_email": {
"type": "string",
"empty": False,
},
"commit_user": {
"type": "string",
"empty": False,
},
"push_url": {
"type": "string",
"required": True,
"empty": False,
},
"push_branch": {
"type": "string",
"empty": False,
},
},
},

2
src/ahriman/core/report/email.py
View File

@ -120,6 +120,6 @@ class Email(Report, JinjaTemplate):
text = self.make_html(result, self.template)
attachments = {}
if self.template_full is not None:
attachments = {"index.html": self.make_html(Result(success=packages), self.template_full)}
attachments["index.html"] = self.make_html(Result(success=packages), self.template_full)
self._send(text, attachments)

6
src/ahriman/core/report/jinja_template.py
View File

@ -76,12 +76,10 @@ class JinjaTemplate:
"""
self.templates = configuration.getpathlist(section, "templates", fallback=[])
self.link_path = configuration.get(section, "link_path")
# base template vars
self.homepage = configuration.get(section, "homepage", fallback=None)
self.link_path = configuration.get(section, "link_path")
self.name = repository_id.name
self.sign_targets, self.default_pgp_key = GPG.sign_options(configuration)
def make_html(self, result: Result, template_name: Path | str) -> str:
@ -99,7 +97,7 @@ class JinjaTemplate:
# idea comes from https://stackoverflow.com/a/38642558
loader = jinja2.FileSystemLoader(searchpath=templates)
environment = jinja2.Environment(loader=loader, autoescape=True)
environment = jinja2.Environment(trim_blocks=True, lstrip_blocks=True, autoescape=True, loader=loader)
template = environment.get_template(template_name)
content = [

32
src/ahriman/core/report/report_trigger.py
View File

@ -40,7 +40,10 @@ class ReportTrigger(Trigger):
"target": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
"schema": {
"type": "string",
"empty": False,
},
},
},
},
@ -70,18 +73,22 @@ class ReportTrigger(Trigger):
"excludes": ["template_full"],
"required": True,
"path_exists": True,
"path_type": "file",
},
"homepage": {
"type": "string",
"empty": False,
"is_url": ["http", "https"],
},
"host": {
"type": "string",
"required": True,
"empty": False,
},
"link_path": {
"type": "string",
"required": True,
"empty": False,
"is_url": [],
},
"no_empty_report": {
@ -90,6 +97,7 @@ class ReportTrigger(Trigger):
},
"password": {
"type": "string",
"empty": False,
},
"port": {
"type": "integer",
@ -101,13 +109,17 @@ class ReportTrigger(Trigger):
"receivers": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
"schema": {
"type": "string",
"empty": False,
},
"required": True,
"empty": False,
},
"sender": {
"type": "string",
"required": True,
"empty": False,
},
"ssl": {
"type": "string",
@ -133,6 +145,7 @@ class ReportTrigger(Trigger):
"excludes": ["template"],
"required": True,
"path_exists": True,
"path_type": "file",
},
"templates": {
"type": "list",
@ -141,10 +154,13 @@ class ReportTrigger(Trigger):
"type": "path",
"coerce": "absolute_path",
"path_exists": True,
"path_type": "dir",
},
"empty": False,
},
"user": {
"type": "string",
"empty": False,
},
},
},
@ -157,11 +173,13 @@ class ReportTrigger(Trigger):
},
"homepage": {
"type": "string",
"empty": False,
"is_url": ["http", "https"],
},
"link_path": {
"type": "string",
"required": True,
"empty": False,
"is_url": [],
},
"path": {
@ -182,6 +200,7 @@ class ReportTrigger(Trigger):
"excludes": ["template"],
"required": True,
"path_exists": True,
"path_type": "file",
},
"templates": {
"type": "list",
@ -190,7 +209,9 @@ class ReportTrigger(Trigger):
"type": "path",
"coerce": "absolute_path",
"path_exists": True,
"path_type": "dir",
},
"empty": False,
},
},
},
@ -204,18 +225,22 @@ class ReportTrigger(Trigger):
"api_key": {
"type": "string",
"required": True,
"empty": False,
},
"chat_id": {
"type": "string",
"required": True,
"empty": False,
},
"homepage": {
"type": "string",
"empty": False,
"is_url": ["http", "https"],
},
"link_path": {
"type": "string",
"required": True,
"empty": False,
"is_url": [],
},
"template": {
@ -231,6 +256,7 @@ class ReportTrigger(Trigger):
"excludes": ["template"],
"required": True,
"path_exists": True,
"path_type": "file",
},
"template_type": {
"type": "string",
@ -243,7 +269,9 @@ class ReportTrigger(Trigger):
"type": "path",
"coerce": "absolute_path",
"path_exists": True,
"path_type": "dir",
},
"empty": False,
},
"timeout": {
"type": "integer",

24
src/ahriman/core/support/keyring_trigger.py
View File

@ -43,7 +43,10 @@ class KeyringTrigger(Trigger):
"target": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
"schema": {
"type": "string",
"empty": False,
},
},
},
},
@ -56,28 +59,47 @@ class KeyringTrigger(Trigger):
},
"description": {
"type": "string",
"empty": False,
},
"homepage": {
"type": "string",
"empty": False,
},
"license": {
"type": "list",
"coerce": "list",
"schema": {
"type": "string",
"empty": False,
},
},
"package": {
"type": "string",
"empty": False,
},
"packagers": {
"type": "list",
"coerce": "list",
"schema": {
"type": "string",
"empty": False,
},
},
"revoked": {
"type": "list",
"coerce": "list",
"schema": {
"type": "string",
"empty": False,
},
},
"trusted": {
"type": "list",
"coerce": "list",
"schema": {
"type": "string",
"empty": False,
},
},
},
},

17
src/ahriman/core/support/mirrorlist_trigger.py
View File

@ -39,7 +39,10 @@ class MirrorlistTrigger(Trigger):
"target": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
"schema": {
"type": "string",
"empty": False,
},
},
},
},
@ -52,16 +55,23 @@ class MirrorlistTrigger(Trigger):
},
"description": {
"type": "string",
"empty": False,
},
"homepage": {
"type": "string",
"empty": False,
},
"license": {
"type": "list",
"coerce": "list",
"schema": {
"type": "string",
"empty": False,
},
},
"package": {
"type": "string",
"empty": False,
},
"path": {
"type": "path",
@ -70,7 +80,12 @@ class MirrorlistTrigger(Trigger):
"servers": {
"type": "list",
"coerce": "list",
"schema": {
"type": "string",
"empty": False,
},
"required": True,
"empty": False,
},
},
},

20
src/ahriman/core/upload/upload_trigger.py
View File

@ -40,7 +40,10 @@ class UploadTrigger(Trigger):
"target": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
"schema": {
"type": "string",
"empty": False,
},
},
},
},
@ -54,13 +57,16 @@ class UploadTrigger(Trigger):
"owner": {
"type": "string",
"required": True,
"empty": False,
},
"password": {
"type": "string",
"empty": False,
},
"repository": {
"type": "string",
"required": True,
"empty": False,
},
"timeout": {
"type": "integer",
@ -73,6 +79,7 @@ class UploadTrigger(Trigger):
},
"username": {
"type": "string",
"empty": False,
},
},
},
@ -86,13 +93,17 @@ class UploadTrigger(Trigger):
"command": {
"type": "list",
"coerce": "list",
"schema": {"type": "string"},
"schema": {
"type": "string",
"empty": False,
},
"required": True,
"empty": False,
},
"remote": {
"type": "string",
"required": True,
"empty": False,
},
},
},
@ -120,10 +131,12 @@ class UploadTrigger(Trigger):
"access_key": {
"type": "string",
"required": True,
"empty": False,
},
"bucket": {
"type": "string",
"required": True,
"empty": False,
},
"chunk_size": {
"type": "integer",
@ -132,14 +145,17 @@ class UploadTrigger(Trigger):
},
"object_path": {
"type": "string",
"empty": False,
},
"region": {
"type": "string",
"required": True,
"empty": False,
},
"secret_key": {
"type": "string",
"required": True,
"empty": False,
},
},
},

4
src/ahriman/web/web.py
View File

@ -147,8 +147,8 @@ def setup_service(repository_id: RepositoryId, configuration: Configuration, spa
setup_cors(application)
application.logger.info("setup templates")
templates = configuration.getpathlist("web", "templates")
aiohttp_jinja2.setup(application, loader=jinja2.FileSystemLoader(searchpath=templates))
loader = jinja2.FileSystemLoader(searchpath=configuration.getpathlist("web", "templates"))
aiohttp_jinja2.setup(application, trim_blocks=True, lstrip_blocks=True, autoescape=True, loader=loader)
application.logger.info("setup configuration")
application["configuration"] = configuration

24
tests/ahriman/core/configuration/test_validator.py
View File

@ -118,3 +118,27 @@ def test_validate_path_exists(validator: Validator, mocker: MockerFixture) -> No
MockCall("field", "Path 2 must not exist"),
MockCall("field", "Path 3 must exist"),
])
def test_validate_path_type(validator: Validator, mocker: MockerFixture) -> None:
"""
must correctly validate path type
"""
error_mock = mocker.patch("ahriman.core.configuration.validator.Validator._error")
mocker.patch("pathlib.Path.is_file", return_value=True)
validator._validate_path_type("file", "field", Path("1"))
mocker.patch("pathlib.Path.is_file", return_value=False)
validator._validate_path_type("file", "field", Path("2"))
mocker.patch("pathlib.Path.is_dir", return_value=True)
validator._validate_path_type("dir", "field", Path("3"))
mocker.patch("pathlib.Path.is_dir", return_value=False)
validator._validate_path_type("dir", "field", Path("4"))
error_mock.assert_has_calls([
MockCall("field", "Path 2 must be type of file"),
MockCall("field", "Path 4 must be type of dir"),
])