feat: improve template processing (#112)

* Improve template processing

* docs update, config validation rules update
This commit is contained in:
Evgenii Alekseev 2023-09-08 23:38:07 +03:00 committed by GitHub
parent a56fe28003
commit 018d9589e1
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
28 changed files with 413 additions and 59 deletions

View File

@ -118,7 +118,7 @@ Web server settings. If any of ``host``/``port`` is not set, web integration wil
* ``password`` - password to authorize in web service in order to update service status, string, required in case if authorization enabled. * ``password`` - password to authorize in web service in order to update service status, string, required in case if authorization enabled.
* ``port`` - port to bind, int, optional. * ``port`` - port to bind, int, optional.
* ``static_path`` - path to directory with static files, string, required. * ``static_path`` - path to directory with static files, string, required.
* ``templates`` - path to templates directory, string, required. * ``templates`` - path to templates directories, space separated list of strings, required.
* ``timeout`` - HTTP request timeout in seconds, int, optional, default is ``30``. * ``timeout`` - HTTP request timeout in seconds, int, optional, default is ``30``.
* ``unix_socket`` - path to the listening unix socket, string, optional. If set, server will create the socket on the specified address which can (and will) be used by application. Note, that unlike usual host/port configuration, unix socket allows to perform requests without authorization. * ``unix_socket`` - path to the listening unix socket, string, optional. If set, server will create the socket on the specified address which can (and will) be used by application. Note, that unlike usual host/port configuration, unix socket allows to perform requests without authorization.
* ``unix_socket_unsafe`` - set unsafe (o+w) permissions to unix socket, boolean, optional, default ``yes``. This option is enabled by default, because it is supposed that unix socket is created in safe environment (only web service is supposed to be used in unsafe), but it can be disabled by configuration. * ``unix_socket_unsafe`` - set unsafe (o+w) permissions to unix socket, boolean, optional, default ``yes``. This option is enabled by default, because it is supposed that unix socket is created in safe environment (only web service is supposed to be used in unsafe), but it can be disabled by configuration.
@ -231,7 +231,6 @@ Section name must be either ``console`` (plus optional architecture name, e.g. `
Section name must be either ``email`` (plus optional architecture name, e.g. ``email:x86_64``) or random name with ``type`` set. Section name must be either ``email`` (plus optional architecture name, e.g. ``email:x86_64``) or random name with ``type`` set.
* ``type`` - type of the report, string, optional, must be set to ``email`` if exists. * ``type`` - type of the report, string, optional, must be set to ``email`` if exists.
* ``full_template_path`` - path to Jinja2 template for full package description index, string, optional.
* ``homepage`` - link to homepage, string, optional. * ``homepage`` - link to homepage, string, optional.
* ``host`` - SMTP host for sending emails, string, required. * ``host`` - SMTP host for sending emails, string, required.
* ``link_path`` - prefix for HTML links, string, required. * ``link_path`` - prefix for HTML links, string, required.
@ -241,7 +240,9 @@ Section name must be either ``email`` (plus optional architecture name, e.g. ``e
* ``receivers`` - SMTP receiver addresses, space separated list of strings, required. * ``receivers`` - SMTP receiver addresses, space separated list of strings, required.
* ``sender`` - SMTP sender address, string, required. * ``sender`` - SMTP sender address, string, required.
* ``ssl`` - SSL mode for SMTP connection, one of ``ssl``, ``starttls``, ``disabled``, optional, default ``disabled``. * ``ssl`` - SSL mode for SMTP connection, one of ``ssl``, ``starttls``, ``disabled``, optional, default ``disabled``.
* ``template_path`` - path to Jinja2 template, string, required. * ``template`` - Jinja2 template name, string, required.
* ``template_full`` - Jinja2 template name for full package description index, string, optional.
* ``templates`` - path to templates directories, space separated list of strings, required.
* ``user`` - SMTP user to authenticate, string, optional. * ``user`` - SMTP user to authenticate, string, optional.
``html`` type ``html`` type
@ -253,7 +254,8 @@ Section name must be either ``html`` (plus optional architecture name, e.g. ``ht
* ``homepage`` - link to homepage, string, optional. * ``homepage`` - link to homepage, string, optional.
* ``link_path`` - prefix for HTML links, string, required. * ``link_path`` - prefix for HTML links, string, required.
* ``path`` - path to html report file, string, required. * ``path`` - path to html report file, string, required.
* ``template_path`` - path to Jinja2 template, string, required. * ``template`` - Jinja2 template name, string, required.
* ``templates`` - path to templates directories, space separated list of strings, required.
``remote-call`` type ``remote-call`` type
^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^
@ -276,8 +278,9 @@ Section name must be either ``telegram`` (plus optional architecture name, e.g.
* ``chat_id`` - telegram chat id, either string with ``@`` or integer value, required. * ``chat_id`` - telegram chat id, either string with ``@`` or integer value, required.
* ``homepage`` - link to homepage, string, optional. * ``homepage`` - link to homepage, string, optional.
* ``link_path`` - prefix for HTML links, string, required. * ``link_path`` - prefix for HTML links, string, required.
* ``template_path`` - path to Jinja2 template, string, required. * ``template`` - Jinja2 template name, string, required.
* ``template_type`` - ``parse_mode`` to be passed to telegram API, one of ``MarkdownV2``, ``HTML``, ``Markdown``, string, optional, default ``HTML``. * ``template_type`` - ``parse_mode`` to be passed to telegram API, one of ``MarkdownV2``, ``HTML``, ``Markdown``, string, optional, default ``HTML``.
* ``templates`` - path to templates directories, space separated list of strings, required.
* ``timeout`` - HTTP request timeout in seconds, int, optional, default is ``30``. * ``timeout`` - HTTP request timeout in seconds, int, optional, default is ``30``.
``upload`` group ``upload`` group

View File

@ -1299,7 +1299,9 @@ The application uses java concept to log messages, e.g. class ``Application`` im
Html customization Html customization
^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^
It is possible to customize html templates. In order to do so, create files somewhere (refer to Jinja2 documentation and the service source code for available parameters) and put ``template_path`` to configuration pointing to this directory. It is possible to customize html templates. In order to do so, create files somewhere (refer to Jinja2 documentation and the service source code for available parameters) and prepend ``templates`` with value pointing to this directory.
In addition, default html templates supports style customization out-of-box. In order to customize style, just put file named ``user-style.jinja2`` to the templates directory.
I did not find my question I did not find my question
^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^

View File

@ -54,14 +54,17 @@ use_utf = yes
[email] [email]
no_empty_report = yes no_empty_report = yes
template_path = /usr/share/ahriman/templates/email-index.jinja2 template = email-index.jinja2
templates = /usr/share/ahriman/templates
ssl = disabled ssl = disabled
[html] [html]
template_path = /usr/share/ahriman/templates/repo-index.jinja2 template = repo-index.jinja2
templates = /usr/share/ahriman/templates
[telegram] [telegram]
template_path = /usr/share/ahriman/templates/telegram-index.jinja2 template = telegram-index.jinja2
templates = /usr/share/ahriman/templates
[upload] [upload]
target = target =

View File

@ -8,6 +8,7 @@
<link rel="shortcut icon" href="/static/favicon.ico"> <link rel="shortcut icon" href="/static/favicon.ico">
{% include "utils/style.jinja2" %} {% include "utils/style.jinja2" %}
{% include "user-style.jinja2" ignore missing %}
</head> </head>
<body> <body>

View File

@ -7,6 +7,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
{% include "utils/style.jinja2" %} {% include "utils/style.jinja2" %}
{% include "user-style.jinja2" ignore missing %}
</head> </head>
<body> <body>

View File

@ -8,6 +8,7 @@
<link rel="shortcut icon" href="/static/favicon.ico"> <link rel="shortcut icon" href="/static/favicon.ico">
{% include "utils/style.jinja2" %} {% include "utils/style.jinja2" %}
{% include "user-style.jinja2" ignore missing %}
</head> </head>
<body> <body>

View File

@ -6,6 +6,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1">
{% include "utils/style.jinja2" %} {% include "utils/style.jinja2" %}
{% include "user-style.jinja2" ignore missing %}
</head> </head>
<body> <body>

View File

@ -82,6 +82,7 @@ class Configuration(configparser.RawConfigParser):
converters={ converters={
"list": shlex.split, "list": shlex.split,
"path": self._convert_path, "path": self._convert_path,
"pathlist": lambda value: [self._convert_path(element) for element in shlex.split(value)],
} }
) )
@ -241,6 +242,8 @@ class Configuration(configparser.RawConfigParser):
def getpath(self, *args: Any, **kwargs: Any) -> Path: ... # type: ignore[empty-body] def getpath(self, *args: Any, **kwargs: Any) -> Path: ... # type: ignore[empty-body]
def getpathlist(self, *args: Any, **kwargs: Any) -> list[Path]: ... # type: ignore[empty-body]
def gettype(self, section: str, repository_id: RepositoryId, *, fallback: str | None = None) -> tuple[str, str]: def gettype(self, section: str, repository_id: RepositoryId, *, fallback: str | None = None) -> tuple[str, str]:
""" """
get type variable with fallback to old logic. Despite the fact that it has same semantics as other get* methods, get type variable with fallback to old logic. Despite the fact that it has same semantics as other get* methods,

View File

@ -44,12 +44,14 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"coerce": "absolute_path", "coerce": "absolute_path",
"required": True, "required": True,
"path_exists": True, "path_exists": True,
"path_type": "dir",
}, },
"logging": { "logging": {
"type": "path", "type": "path",
"coerce": "absolute_path", "coerce": "absolute_path",
"required": True, "required": True,
"path_exists": True, "path_exists": True,
"path_type": "file",
}, },
"suppress_http_log_errors": { "suppress_http_log_errors": {
"type": "boolean", "type": "boolean",
@ -68,12 +70,16 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"mirror": { "mirror": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
"is_url": [], "is_url": [],
}, },
"repositories": { "repositories": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {"type": "string"}, "schema": {
"type": "string",
"empty": False,
},
"required": True, "required": True,
"empty": False, "empty": False,
}, },
@ -82,6 +88,7 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"coerce": "absolute_path", "coerce": "absolute_path",
"required": True, "required": True,
"path_exists": True, "path_exists": True,
"path_type": "dir",
}, },
"use_ahriman_cache": { "use_ahriman_cache": {
"type": "boolean", "type": "boolean",
@ -113,9 +120,11 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
}, },
"client_id": { "client_id": {
"type": "string", "type": "string",
"empty": False,
}, },
"client_secret": { "client_secret": {
"type": "string", "type": "string",
"empty": False,
}, },
"cookie_secret_key": { "cookie_secret_key": {
"type": "string", "type": "string",
@ -129,9 +138,11 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
}, },
"oauth_provider": { "oauth_provider": {
"type": "string", "type": "string",
"empty": False,
}, },
"oauth_scopes": { "oauth_scopes": {
"type": "string", "type": "string",
"empty": False,
}, },
"salt": { "salt": {
"type": "string", "type": "string",
@ -144,36 +155,55 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"archbuild_flags": { "archbuild_flags": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {"type": "string"}, "schema": {
"type": "string",
"empty": False,
},
}, },
"build_command": { "build_command": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
}, },
"ignore_packages": { "ignore_packages": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {"type": "string"}, "schema": {
"type": "string",
"empty": False,
},
}, },
"makepkg_flags": { "makepkg_flags": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {"type": "string"}, "schema": {
"type": "string",
"empty": False,
},
}, },
"makechrootpkg_flags": { "makechrootpkg_flags": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {"type": "string"}, "schema": {
"type": "string",
"empty": False,
},
}, },
"triggers": { "triggers": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {"type": "string"}, "schema": {
"type": "string",
"empty": False,
},
}, },
"triggers_known": { "triggers_known": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {"type": "string"}, "schema": {
"type": "string",
"empty": False,
},
}, },
"vcs_allowed_age": { "vcs_allowed_age": {
"type": "integer", "type": "integer",
@ -187,10 +217,14 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"schema": { "schema": {
"name": { "name": {
"type": "string", "type": "string",
"empty": False,
}, },
"root": { "root": {
"type": "string", "type": "path",
"coerce": "absolute_path",
"required": True, "required": True,
"path_exists": True,
"path_type": "dir",
}, },
}, },
}, },
@ -208,6 +242,7 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
}, },
"key": { "key": {
"type": "string", "type": "string",
"empty": False,
}, },
}, },
}, },
@ -216,6 +251,7 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"schema": { "schema": {
"address": { "address": {
"type": "string", "type": "string",
"empty": False,
"is_url": ["http", "https"], "is_url": ["http", "https"],
}, },
"debug": { "debug": {
@ -229,7 +265,10 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"debug_allowed_hosts": { "debug_allowed_hosts": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {"type": "string"}, "schema": {
"type": "string",
"empty": False,
},
}, },
"enable_archive_upload": { "enable_archive_upload": {
"type": "boolean", "type": "boolean",
@ -237,10 +276,12 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
}, },
"host": { "host": {
"type": "string", "type": "string",
"empty": False,
"is_ip_address": ["localhost"], "is_ip_address": ["localhost"],
}, },
"index_url": { "index_url": {
"type": "string", "type": "string",
"empty": False,
"is_url": ["http", "https"], "is_url": ["http", "https"],
}, },
"max_body_size": { "max_body_size": {
@ -250,6 +291,7 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
}, },
"password": { "password": {
"type": "string", "type": "string",
"empty": False,
}, },
"port": { "port": {
"type": "integer", "type": "integer",
@ -262,12 +304,18 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"coerce": "absolute_path", "coerce": "absolute_path",
"required": True, "required": True,
"path_exists": True, "path_exists": True,
"path_type": "dir",
}, },
"templates": { "templates": {
"type": "list",
"coerce": "list",
"schema": {
"type": "path", "type": "path",
"coerce": "absolute_path", "coerce": "absolute_path",
"required": True,
"path_exists": True, "path_exists": True,
"path_type": "dir",
},
"empty": False,
}, },
"timeout": { "timeout": {
"type": "integer", "type": "integer",
@ -284,6 +332,7 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
}, },
"username": { "username": {
"type": "string", "type": "string",
"empty": False,
}, },
"wait_timeout": { "wait_timeout": {
"type": "integer", "type": "integer",

View File

@ -162,3 +162,21 @@ class Validator(RootValidator):
self._error(field, f"Path {value} must not exist") self._error(field, f"Path {value} must not exist")
case False if constraint: case False if constraint:
self._error(field, f"Path {value} must exist") self._error(field, f"Path {value} must exist")
def _validate_path_type(self, constraint: str, field: str, value: Path) -> None:
"""
check if paths is file, directory or whatever. The match will be performed as call of ``is_{constraint}``
method of the path object
Args:
constraint(str): path type to be matched
field(str): field name to be checked
value(Path): value to be checked
Examples:
The rule's arguments are validated against this schema:
{"type": "string"}
"""
fn = getattr(value, f"is_{constraint}")
if not fn():
self._error(field, f"Path {value} must be type of {constraint}")

View File

@ -38,7 +38,10 @@ class RemotePullTrigger(Trigger):
"target": { "target": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {"type": "string"}, "schema": {
"type": "string",
"empty": False,
},
}, },
}, },
}, },
@ -48,9 +51,11 @@ class RemotePullTrigger(Trigger):
"pull_url": { "pull_url": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
}, },
"pull_branch": { "pull_branch": {
"type": "string", "type": "string",
"empty": False,
}, },
}, },
}, },

View File

@ -43,7 +43,10 @@ class RemotePushTrigger(Trigger):
"target": { "target": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {"type": "string"}, "schema": {
"type": "string",
"empty": False,
},
}, },
}, },
}, },
@ -52,16 +55,20 @@ class RemotePushTrigger(Trigger):
"schema": { "schema": {
"commit_email": { "commit_email": {
"type": "string", "type": "string",
"empty": False,
}, },
"commit_user": { "commit_user": {
"type": "string", "type": "string",
"empty": False,
}, },
"push_url": { "push_url": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
}, },
"push_branch": { "push_branch": {
"type": "string", "type": "string",
"empty": False,
}, },
}, },
}, },

View File

@ -37,7 +37,6 @@ class Email(Report, JinjaTemplate):
email report generator email report generator
Attributes: Attributes:
full_template_path(Path): path to template for full package list
host(str): SMTP host to connect host(str): SMTP host to connect
no_empty_report(bool): skip empty report generation no_empty_report(bool): skip empty report generation
password(str | None): password to authenticate via SMTP password(str | None): password to authenticate via SMTP
@ -45,7 +44,8 @@ class Email(Report, JinjaTemplate):
receivers(list[str]): list of receivers emails receivers(list[str]): list of receivers emails
sender(str): sender email address sender(str): sender email address
ssl(SmtpSSLSettings): SSL mode for SMTP connection ssl(SmtpSSLSettings): SSL mode for SMTP connection
template_path(Path): path to template for built packages template(Path | str): path or name to template for built packages
template_full(Path | str | None): path or name to template for full package list
user(str | None): username to authenticate via SMTP user(str | None): username to authenticate via SMTP
""" """
@ -61,8 +61,10 @@ class Email(Report, JinjaTemplate):
Report.__init__(self, repository_id, configuration) Report.__init__(self, repository_id, configuration)
JinjaTemplate.__init__(self, repository_id, configuration, section) JinjaTemplate.__init__(self, repository_id, configuration, section)
self.full_template_path = configuration.getpath(section, "full_template_path", fallback=None) self.template = configuration.get(section, "template", fallback=None) or \
self.template_path = configuration.getpath(section, "template_path") configuration.getpath(section, "template_path")
self.template_full = configuration.get(section, "template_full", fallback=None) or \
configuration.getpath(section, "full_template_path", fallback=None)
# base smtp settings # base smtp settings
self.host = configuration.get(section, "host") self.host = configuration.get(section, "host")
@ -114,9 +116,10 @@ class Email(Report, JinjaTemplate):
""" """
if self.no_empty_report and not result.success: if self.no_empty_report and not result.success:
return return
text = self.make_html(result, self.template_path)
if self.full_template_path is not None: text = self.make_html(result, self.template)
attachments = {"index.html": self.make_html(Result(success=packages), self.full_template_path)}
else:
attachments = {} attachments = {}
if self.template_full is not None:
attachments["index.html"] = self.make_html(Result(success=packages), self.template_full)
self._send(text, attachments) self._send(text, attachments)

View File

@ -31,7 +31,7 @@ class HTML(Report, JinjaTemplate):
Attributes: Attributes:
report_path(Path): output path to html report report_path(Path): output path to html report
template_path(Path): path to template for full package list template(Path | str): name or path to template for full package list
""" """
def __init__(self, repository_id: RepositoryId, configuration: Configuration, section: str) -> None: def __init__(self, repository_id: RepositoryId, configuration: Configuration, section: str) -> None:
@ -47,7 +47,8 @@ class HTML(Report, JinjaTemplate):
JinjaTemplate.__init__(self, repository_id, configuration, section) JinjaTemplate.__init__(self, repository_id, configuration, section)
self.report_path = configuration.getpath(section, "path") self.report_path = configuration.getpath(section, "path")
self.template_path = configuration.getpath(section, "template_path") self.template = configuration.get(section, "template", fallback=None) or \
configuration.getpath(section, "template_path")
def generate(self, packages: list[Package], result: Result) -> None: def generate(self, packages: list[Package], result: Result) -> None:
""" """
@ -57,5 +58,5 @@ class HTML(Report, JinjaTemplate):
packages(list[Package]): list of packages to generate report packages(list[Package]): list of packages to generate report
result(Result): build result result(Result): build result
""" """
html = self.make_html(Result(success=packages), self.template_path) html = self.make_html(Result(success=packages), self.template)
self.report_path.write_text(html, encoding="utf8") self.report_path.write_text(html, encoding="utf8")

View File

@ -57,11 +57,12 @@ class JinjaTemplate:
* repository - repository name, string, required * repository - repository name, string, required
Attributes: Attributes:
default_pgp_key(str | None): default PGP key
homepage(str | None): homepage link if any (for footer) homepage(str | None): homepage link if any (for footer)
link_path(str): prefix fo packages to download link_path(str): prefix fo packages to download
name(str): repository name name(str): repository name
default_pgp_key(str | None): default PGP key
sign_targets(set[SignSettings]): targets to sign enabled in configuration sign_targets(set[SignSettings]): targets to sign enabled in configuration
templates(list[Path]): list of directories with templates
""" """
def __init__(self, repository_id: RepositoryId, configuration: Configuration, section: str) -> None: def __init__(self, repository_id: RepositoryId, configuration: Configuration, section: str) -> None:
@ -73,26 +74,31 @@ class JinjaTemplate:
configuration(Configuration): configuration instance configuration(Configuration): configuration instance
section(str): settings section name section(str): settings section name
""" """
self.link_path = configuration.get(section, "link_path") self.templates = configuration.getpathlist(section, "templates", fallback=[])
# base template vars # base template vars
self.homepage = configuration.get(section, "homepage", fallback=None) self.homepage = configuration.get(section, "homepage", fallback=None)
self.link_path = configuration.get(section, "link_path")
self.name = repository_id.name self.name = repository_id.name
self.sign_targets, self.default_pgp_key = GPG.sign_options(configuration) self.sign_targets, self.default_pgp_key = GPG.sign_options(configuration)
def make_html(self, result: Result, template_path: Path) -> str: def make_html(self, result: Result, template_name: Path | str) -> str:
""" """
generate report for the specified packages generate report for the specified packages
Args: Args:
result(Result): build result result(Result): build result
template_path(Path): path to jinja template template_name(Path | str): name of the template or path to it (legacy configuration)
""" """
templates = self.templates[:]
if isinstance(template_name, Path):
templates.append(template_name.parent)
template_name = template_name.name
# idea comes from https://stackoverflow.com/a/38642558 # idea comes from https://stackoverflow.com/a/38642558
loader = jinja2.FileSystemLoader(searchpath=template_path.parent) loader = jinja2.FileSystemLoader(searchpath=templates)
environment = jinja2.Environment(loader=loader, autoescape=True) environment = jinja2.Environment(trim_blocks=True, lstrip_blocks=True, autoescape=True, loader=loader)
template = environment.get_template(template_path.name) template = environment.get_template(template_name)
content = [ content = [
{ {

View File

@ -40,7 +40,10 @@ class ReportTrigger(Trigger):
"target": { "target": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {"type": "string"}, "schema": {
"type": "string",
"empty": False,
},
}, },
}, },
}, },
@ -67,19 +70,25 @@ class ReportTrigger(Trigger):
"full_template_path": { "full_template_path": {
"type": "path", "type": "path",
"coerce": "absolute_path", "coerce": "absolute_path",
"excludes": ["template_full"],
"required": True,
"path_exists": True, "path_exists": True,
"path_type": "file",
}, },
"homepage": { "homepage": {
"type": "string", "type": "string",
"empty": False,
"is_url": ["http", "https"], "is_url": ["http", "https"],
}, },
"host": { "host": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
}, },
"link_path": { "link_path": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
"is_url": [], "is_url": [],
}, },
"no_empty_report": { "no_empty_report": {
@ -88,6 +97,7 @@ class ReportTrigger(Trigger):
}, },
"password": { "password": {
"type": "string", "type": "string",
"empty": False,
}, },
"port": { "port": {
"type": "integer", "type": "integer",
@ -99,26 +109,58 @@ class ReportTrigger(Trigger):
"receivers": { "receivers": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {"type": "string"}, "schema": {
"type": "string",
"empty": False,
},
"required": True, "required": True,
"empty": False, "empty": False,
}, },
"sender": { "sender": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
}, },
"ssl": { "ssl": {
"type": "string", "type": "string",
"allowed": ["ssl", "starttls", "disabled"], "allowed": ["ssl", "starttls", "disabled"],
}, },
"template": {
"type": "string",
"excludes": ["template_path"],
"dependencies": ["templates"],
"required": True,
"empty": False,
},
"template_full": {
"type": "string",
"excludes": ["template_path"],
"dependencies": ["templates"],
"required": True,
"empty": False,
},
"template_path": { "template_path": {
"type": "path", "type": "path",
"coerce": "absolute_path", "coerce": "absolute_path",
"excludes": ["template"],
"required": True, "required": True,
"path_exists": True, "path_exists": True,
"path_type": "file",
},
"templates": {
"type": "list",
"coerce": "list",
"schema": {
"type": "path",
"coerce": "absolute_path",
"path_exists": True,
"path_type": "dir",
},
"empty": False,
}, },
"user": { "user": {
"type": "string", "type": "string",
"empty": False,
}, },
}, },
}, },
@ -131,11 +173,13 @@ class ReportTrigger(Trigger):
}, },
"homepage": { "homepage": {
"type": "string", "type": "string",
"empty": False,
"is_url": ["http", "https"], "is_url": ["http", "https"],
}, },
"link_path": { "link_path": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
"is_url": [], "is_url": [],
}, },
"path": { "path": {
@ -143,11 +187,31 @@ class ReportTrigger(Trigger):
"coerce": "absolute_path", "coerce": "absolute_path",
"required": True, "required": True,
}, },
"template": {
"type": "string",
"excludes": ["template_path"],
"dependencies": ["templates"],
"required": True,
"empty": False,
},
"template_path": { "template_path": {
"type": "path", "type": "path",
"coerce": "absolute_path", "coerce": "absolute_path",
"excludes": ["template"],
"required": True, "required": True,
"path_exists": True, "path_exists": True,
"path_type": "file",
},
"templates": {
"type": "list",
"coerce": "list",
"schema": {
"type": "path",
"coerce": "absolute_path",
"path_exists": True,
"path_type": "dir",
},
"empty": False,
}, },
}, },
}, },
@ -161,30 +225,54 @@ class ReportTrigger(Trigger):
"api_key": { "api_key": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
}, },
"chat_id": { "chat_id": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
}, },
"homepage": { "homepage": {
"type": "string", "type": "string",
"empty": False,
"is_url": ["http", "https"], "is_url": ["http", "https"],
}, },
"link_path": { "link_path": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
"is_url": [], "is_url": [],
}, },
"template": {
"type": "string",
"excludes": ["template_path"],
"dependencies": ["templates"],
"required": True,
"empty": False,
},
"template_path": { "template_path": {
"type": "path", "type": "path",
"coerce": "absolute_path", "coerce": "absolute_path",
"excludes": ["template"],
"required": True, "required": True,
"path_exists": True, "path_exists": True,
"path_type": "file",
}, },
"template_type": { "template_type": {
"type": "string", "type": "string",
"allowed": ["MarkdownV2", "HTML", "Markdown"], "allowed": ["MarkdownV2", "HTML", "Markdown"],
}, },
"templates": {
"type": "list",
"coerce": "list",
"schema": {
"type": "path",
"coerce": "absolute_path",
"path_exists": True,
"path_type": "dir",
},
"empty": False,
},
"timeout": { "timeout": {
"type": "integer", "type": "integer",
"coerce": "integer", "coerce": "integer",

View File

@ -35,7 +35,7 @@ class Telegram(Report, JinjaTemplate, SyncHttpClient):
TELEGRAM_MAX_CONTENT_LENGTH(int): (class attribute) max content length of the message TELEGRAM_MAX_CONTENT_LENGTH(int): (class attribute) max content length of the message
api_key(str): bot api key api_key(str): bot api key
chat_id(str): chat id to post message, either string with @ or integer chat_id(str): chat id to post message, either string with @ or integer
template_path(Path): path to template for built packages template(Path | str): name or path to template for built packages
template_type(str): template message type to be used in parse mode, one of MarkdownV2, HTML, Markdown template_type(str): template message type to be used in parse mode, one of MarkdownV2, HTML, Markdown
""" """
@ -57,7 +57,8 @@ class Telegram(Report, JinjaTemplate, SyncHttpClient):
self.api_key = configuration.get(section, "api_key") self.api_key = configuration.get(section, "api_key")
self.chat_id = configuration.get(section, "chat_id") self.chat_id = configuration.get(section, "chat_id")
self.template_path = configuration.getpath(section, "template_path") self.template = configuration.get(section, "template", fallback=None) or \
configuration.getpath(section, "template_path")
self.template_type = configuration.get(section, "template_type", fallback="HTML") self.template_type = configuration.get(section, "template_type", fallback="HTML")
def _send(self, text: str) -> None: def _send(self, text: str) -> None:
@ -83,7 +84,7 @@ class Telegram(Report, JinjaTemplate, SyncHttpClient):
""" """
if not result.success: if not result.success:
return return
text = self.make_html(result, self.template_path) text = self.make_html(result, self.template)
# telegram content is limited by 4096 symbols, so we are going to split the message by new lines # telegram content is limited by 4096 symbols, so we are going to split the message by new lines
# to fit into this restriction # to fit into this restriction
while len(text) > self.TELEGRAM_MAX_CONTENT_LENGTH: while len(text) > self.TELEGRAM_MAX_CONTENT_LENGTH:

View File

@ -43,7 +43,10 @@ class KeyringTrigger(Trigger):
"target": { "target": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {"type": "string"}, "schema": {
"type": "string",
"empty": False,
},
}, },
}, },
}, },
@ -56,28 +59,47 @@ class KeyringTrigger(Trigger):
}, },
"description": { "description": {
"type": "string", "type": "string",
"empty": False,
}, },
"homepage": { "homepage": {
"type": "string", "type": "string",
"empty": False,
}, },
"license": { "license": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {
"type": "string",
"empty": False,
},
}, },
"package": { "package": {
"type": "string", "type": "string",
"empty": False,
}, },
"packagers": { "packagers": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {
"type": "string",
"empty": False,
},
}, },
"revoked": { "revoked": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {
"type": "string",
"empty": False,
},
}, },
"trusted": { "trusted": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {
"type": "string",
"empty": False,
},
}, },
}, },
}, },

View File

@ -39,7 +39,10 @@ class MirrorlistTrigger(Trigger):
"target": { "target": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {"type": "string"}, "schema": {
"type": "string",
"empty": False,
},
}, },
}, },
}, },
@ -52,16 +55,23 @@ class MirrorlistTrigger(Trigger):
}, },
"description": { "description": {
"type": "string", "type": "string",
"empty": False,
}, },
"homepage": { "homepage": {
"type": "string", "type": "string",
"empty": False,
}, },
"license": { "license": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {
"type": "string",
"empty": False,
},
}, },
"package": { "package": {
"type": "string", "type": "string",
"empty": False,
}, },
"path": { "path": {
"type": "path", "type": "path",
@ -70,7 +80,12 @@ class MirrorlistTrigger(Trigger):
"servers": { "servers": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {
"type": "string",
"empty": False,
},
"required": True, "required": True,
"empty": False,
}, },
}, },
}, },

View File

@ -40,7 +40,10 @@ class UploadTrigger(Trigger):
"target": { "target": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {"type": "string"}, "schema": {
"type": "string",
"empty": False,
},
}, },
}, },
}, },
@ -54,13 +57,16 @@ class UploadTrigger(Trigger):
"owner": { "owner": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
}, },
"password": { "password": {
"type": "string", "type": "string",
"empty": False,
}, },
"repository": { "repository": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
}, },
"timeout": { "timeout": {
"type": "integer", "type": "integer",
@ -73,6 +79,7 @@ class UploadTrigger(Trigger):
}, },
"username": { "username": {
"type": "string", "type": "string",
"empty": False,
}, },
}, },
}, },
@ -86,13 +93,17 @@ class UploadTrigger(Trigger):
"command": { "command": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
"schema": {"type": "string"}, "schema": {
"type": "string",
"empty": False,
},
"required": True, "required": True,
"empty": False, "empty": False,
}, },
"remote": { "remote": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
}, },
}, },
}, },
@ -120,10 +131,12 @@ class UploadTrigger(Trigger):
"access_key": { "access_key": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
}, },
"bucket": { "bucket": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
}, },
"chunk_size": { "chunk_size": {
"type": "integer", "type": "integer",
@ -132,14 +145,17 @@ class UploadTrigger(Trigger):
}, },
"object_path": { "object_path": {
"type": "string", "type": "string",
"empty": False,
}, },
"region": { "region": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
}, },
"secret_key": { "secret_key": {
"type": "string", "type": "string",
"required": True, "required": True,
"empty": False,
}, },
}, },
}, },

View File

@ -147,7 +147,8 @@ def setup_service(repository_id: RepositoryId, configuration: Configuration, spa
setup_cors(application) setup_cors(application)
application.logger.info("setup templates") application.logger.info("setup templates")
aiohttp_jinja2.setup(application, loader=jinja2.FileSystemLoader(configuration.getpath("web", "templates"))) loader = jinja2.FileSystemLoader(searchpath=configuration.getpathlist("web", "templates"))
aiohttp_jinja2.setup(application, trim_blocks=True, lstrip_blocks=True, autoescape=True, loader=loader)
application.logger.info("setup configuration") application.logger.info("setup configuration")
application["configuration"] = configuration application["configuration"] = configuration

View File

@ -229,6 +229,19 @@ def test_getpath_without_fallback(configuration: Configuration) -> None:
assert configuration.getpath("build", "option") assert configuration.getpath("build", "option")
def test_getpathlist(configuration: Configuration) -> None:
"""
must extract path list
"""
path = Path("/a/b/c")
configuration.set_option("build", "path", f"""{path} {path.relative_to("/")}""")
result = configuration.getpathlist("build", "path")
assert all(element.is_absolute() for element in result)
assert path in result
assert all(element.is_relative_to("/") for element in result)
def test_gettype(configuration: Configuration) -> None: def test_gettype(configuration: Configuration) -> None:
""" """
must extract type from variable must extract type from variable

View File

@ -118,3 +118,27 @@ def test_validate_path_exists(validator: Validator, mocker: MockerFixture) -> No
MockCall("field", "Path 2 must not exist"), MockCall("field", "Path 2 must not exist"),
MockCall("field", "Path 3 must exist"), MockCall("field", "Path 3 must exist"),
]) ])
def test_validate_path_type(validator: Validator, mocker: MockerFixture) -> None:
"""
must correctly validate path type
"""
error_mock = mocker.patch("ahriman.core.configuration.validator.Validator._error")
mocker.patch("pathlib.Path.is_file", return_value=True)
validator._validate_path_type("file", "field", Path("1"))
mocker.patch("pathlib.Path.is_file", return_value=False)
validator._validate_path_type("file", "field", Path("2"))
mocker.patch("pathlib.Path.is_dir", return_value=True)
validator._validate_path_type("dir", "field", Path("3"))
mocker.patch("pathlib.Path.is_dir", return_value=False)
validator._validate_path_type("dir", "field", Path("4"))
error_mock.assert_has_calls([
MockCall("field", "Path 2 must be type of file"),
MockCall("field", "Path 4 must be type of dir"),
])

View File

@ -8,6 +8,35 @@ from ahriman.models.package import Package
from ahriman.models.result import Result from ahriman.models.result import Result
def test_template(configuration: Configuration) -> None:
"""
must correctly parse template name and path
"""
template = configuration.get("email", "template")
root, repository_id = configuration.check_loaded()
assert Email(repository_id, configuration, "email").template == template
configuration.remove_option("email", "template")
configuration.set_option("email", "template_path", template)
assert Email(repository_id, configuration, "email").template == root.parent / template
def test_template_full(configuration: Configuration) -> None:
"""
must correctly parse template name and path
"""
template = "template"
root, repository_id = configuration.check_loaded()
configuration.set_option("email", "template_full", template)
assert Email(repository_id, configuration, "email").template_full == template
configuration.remove_option("email", "template_full")
configuration.set_option("email", "full_template_path", template)
assert Email(repository_id, configuration, "email").template_full == root.parent / template
def test_send(email: Email, mocker: MockerFixture) -> None: def test_send(email: Email, mocker: MockerFixture) -> None:
""" """
must send an email with attachment must send an email with attachment
@ -115,7 +144,7 @@ def test_generate_with_built_and_full_path(email: Email, package_ahriman: Packag
""" """
send_mock = mocker.patch("ahriman.core.report.email.Email._send") send_mock = mocker.patch("ahriman.core.report.email.Email._send")
email.full_template_path = email.template_path email.template_full = email.template
email.generate([package_ahriman], result) email.generate([package_ahriman], result)
send_mock.assert_called_once_with(pytest.helpers.anyvar(int), pytest.helpers.anyvar(int)) send_mock.assert_called_once_with(pytest.helpers.anyvar(int), pytest.helpers.anyvar(int))

View File

@ -8,6 +8,20 @@ from ahriman.models.package import Package
from ahriman.models.result import Result from ahriman.models.result import Result
def test_template(configuration: Configuration) -> None:
"""
must correctly parse template name and path
"""
template = configuration.get("html", "template")
root, repository_id = configuration.check_loaded()
assert HTML(repository_id, configuration, "html").template == template
configuration.remove_option("html", "template")
configuration.set_option("html", "template_path", template)
assert HTML(repository_id, configuration, "html").template == root.parent / template
def test_generate(configuration: Configuration, package_ahriman: Package, mocker: MockerFixture) -> None: def test_generate(configuration: Configuration, package_ahriman: Package, mocker: MockerFixture) -> None:
""" """
must generate report must generate report

View File

@ -8,7 +8,17 @@ def test_generate(configuration: Configuration, package_ahriman: Package) -> Non
""" """
must generate html report must generate html report
""" """
path = configuration.getpath("html", "template_path") name = configuration.getpath("html", "template")
_, repository_id = configuration.check_loaded()
report = JinjaTemplate(repository_id, configuration, "html")
assert report.make_html(Result(success=[package_ahriman]), name)
def test_generate_from_path(configuration: Configuration, package_ahriman: Package) -> None:
"""
must generate html report from path
"""
path = configuration.getpath("html", "templates") / configuration.get("html", "template")
_, repository_id = configuration.check_loaded() _, repository_id = configuration.check_loaded()
report = JinjaTemplate(repository_id, configuration, "html") report = JinjaTemplate(repository_id, configuration, "html")
assert report.make_html(Result(success=[package_ahriman]), path) assert report.make_html(Result(success=[package_ahriman]), path)

View File

@ -10,6 +10,20 @@ from ahriman.models.package import Package
from ahriman.models.result import Result from ahriman.models.result import Result
def test_template(configuration: Configuration) -> None:
"""
must correctly parse template name and path
"""
template = configuration.get("telegram", "template")
root, repository_id = configuration.check_loaded()
assert Telegram(repository_id, configuration, "telegram").template == template
configuration.remove_option("telegram", "template")
configuration.set_option("telegram", "template_path", template)
assert Telegram(repository_id, configuration, "telegram").template == root.parent / template
def test_send(telegram: Telegram, mocker: MockerFixture) -> None: def test_send(telegram: Telegram, mocker: MockerFixture) -> None:
""" """
must send a message must send a message

View File

@ -63,7 +63,8 @@ no_empty_report = no
port = 587 port = 587
receivers = mail@example.com receivers = mail@example.com
sender = mail@example.com sender = mail@example.com
template_path = ../web/templates/repo-index.jinja2 template = repo-index.jinja2
templates = ../web/templates
[console] [console]
use_utf = yes use_utf = yes
@ -72,7 +73,8 @@ use_utf = yes
path = path =
homepage = homepage =
link_path = link_path =
template_path = ../web/templates/repo-index.jinja2 template = repo-index.jinja2
templates = ../web/templates
[remote-call] [remote-call]
manual = yes manual = yes
@ -82,7 +84,8 @@ api_key = apikey
chat_id = @ahrimantestchat chat_id = @ahrimantestchat
homepage = homepage =
link_path = link_path =
template_path = ../web/templates/telegram-index.jinja2 template = telegram-index.jinja2
templates = ../web/templates
[upload] [upload]
target = target =