feat: replace scan paths options to single one

It has been found that previous system didn't allow to configure
specific cases (e.g. a whitelisted directory inside /usr/lib/cmake). The
current solution replaces two options to single one, which also allows a
regular expressions

Also PackageArchive class has been moved to core package, because it is
more about service rather than model
This commit is contained in:
Evgenii Alekseev 2024-08-25 16:22:11 +03:00
parent 16308dc3ae
commit 69f0966ff1
19 changed files with 141 additions and 155 deletions

View File

@ -92,7 +92,7 @@ Again, the most checks can be performed by `tox` command, though some additional
``` ```
* Type annotations are the must, even for local functions. For the function argument `self` (for instance methods) and `cls` (for class methods) should not be annotated. * Type annotations are the must, even for local functions. For the function argument `self` (for instance methods) and `cls` (for class methods) should not be annotated.
* For collection types built-in classes must be used if possible (e.g. `dict` instead of `typing.Dict`, `tuple` instead of `typing.Tuple`). In case if built-in type is not available, but `collections.abc` provides interface, it must be used (e.g. `collections.abc.Awaitable` instead of `typing.Awaitable`, `collections.abc.Iterable` instead of `typing.Iterable`). For union classes, the bar operator (`|`) must be used (e.g. `float | int` instead of `typing.Union[float, int]`), which also includes `typinng.Optional` (e.g. `str | None` instead of `Optional[str]`). * For collection types built-in classes must be used if possible (e.g. `dict` instead of `typing.Dict`, `tuple` instead of `typing.Tuple`). In case if built-in type is not available, but `collections.abc` provides interface, it must be used (e.g. `collections.abc.Awaitable` instead of `typing.Awaitable`, `collections.abc.Iterable` instead of `typing.Iterable`). For union classes, the bar operator (`|`) must be used (e.g. `float | int` instead of `typing.Union[float, int]`), which also includes `typing.Optional` (e.g. `str | None` instead of `Optional[str]`).
* `classmethod` should (almost) always return `Self`. In case of mypy warning (e.g. if there is a branch in which function doesn't return the instance of `cls`) consider using `staticmethod` instead. * `classmethod` should (almost) always return `Self`. In case of mypy warning (e.g. if there is a branch in which function doesn't return the instance of `cls`) consider using `staticmethod` instead.
* Recommended order of function definitions in class: * Recommended order of function definitions in class:

View File

@ -2,7 +2,7 @@
[![tests status](https://github.com/arcan1s/ahriman/actions/workflows/tests.yml/badge.svg)](https://github.com/arcan1s/ahriman/actions/workflows/tests.yml) [![tests status](https://github.com/arcan1s/ahriman/actions/workflows/tests.yml/badge.svg)](https://github.com/arcan1s/ahriman/actions/workflows/tests.yml)
[![setup status](https://github.com/arcan1s/ahriman/actions/workflows/setup.yml/badge.svg)](https://github.com/arcan1s/ahriman/actions/workflows/setup.yml) [![setup status](https://github.com/arcan1s/ahriman/actions/workflows/setup.yml/badge.svg)](https://github.com/arcan1s/ahriman/actions/workflows/setup.yml)
[![Docker Image Version (latest semver)](https://img.shields.io/docker/v/arcan1s/ahriman?label=Docker%20image)](https://hub.docker.com/r/arcan1s/ahriman) [![Docker Image Version (latest semver)](https://img.shields.io/docker/v/arcan1s/ahriman?label=Docker%20image&sort=semver)](https://hub.docker.com/r/arcan1s/ahriman)
[![CodeFactor](https://www.codefactor.io/repository/github/arcan1s/ahriman/badge)](https://www.codefactor.io/repository/github/arcan1s/ahriman) [![CodeFactor](https://www.codefactor.io/repository/github/arcan1s/ahriman/badge)](https://www.codefactor.io/repository/github/arcan1s/ahriman)
[![Documentation Status](https://readthedocs.org/projects/ahriman/badge/?version=latest)](https://ahriman.readthedocs.io) [![Documentation Status](https://readthedocs.org/projects/ahriman/badge/?version=latest)](https://ahriman.readthedocs.io)

View File

@ -4,6 +4,14 @@ ahriman.core.build\_tools package
Submodules Submodules
---------- ----------
ahriman.core.build\_tools.package\_archive module
-------------------------------------------------
.. automodule:: ahriman.core.build_tools.package_archive
:members:
:no-undoc-members:
:show-inheritance:
ahriman.core.build\_tools.sources module ahriman.core.build\_tools.sources module
---------------------------------------- ----------------------------------------

View File

@ -124,14 +124,6 @@ ahriman.models.package module
:no-undoc-members: :no-undoc-members:
:show-inheritance: :show-inheritance:
ahriman.models.package\_archive module
--------------------------------------
.. automodule:: ahriman.models.package_archive
:members:
:no-undoc-members:
:show-inheritance:
ahriman.models.package\_description module ahriman.models.package\_description module
------------------------------------------ ------------------------------------------

View File

@ -81,14 +81,13 @@ Authorized users are stored inside internal database, if any of external provide
Build related configuration. Group name can refer to architecture, e.g. ``build:x86_64`` can be used for x86_64 architecture specific settings. Build related configuration. Group name can refer to architecture, e.g. ``build:x86_64`` can be used for x86_64 architecture specific settings.
* ``allowed_scan_paths`` - paths to be used for implicit dependencies scan, scape separated list of paths, optional.
* ``archbuild_flags`` - additional flags passed to ``archbuild`` command, space separated list of strings, optional. * ``archbuild_flags`` - additional flags passed to ``archbuild`` command, space separated list of strings, optional.
* ``blacklisted_scan_paths`` - paths to be excluded for implicit dependencies scan, scape separated list of paths, optional. Normally all elements of this option must be child paths of any of ``allowed_scan_paths`` element.
* ``build_command`` - default build command, string, required. * ``build_command`` - default build command, string, required.
* ``ignore_packages`` - list packages to ignore during a regular update (manual update will still work), space separated list of strings, optional. * ``ignore_packages`` - list packages to ignore during a regular update (manual update will still work), space separated list of strings, optional.
* ``include_debug_packages`` - distribute debug packages, boolean, optional, default ``yes``. * ``include_debug_packages`` - distribute debug packages, boolean, optional, default ``yes``.
* ``makepkg_flags`` - additional flags passed to ``makepkg`` command, space separated list of strings, optional. * ``makepkg_flags`` - additional flags passed to ``makepkg`` command, space separated list of strings, optional.
* ``makechrootpkg_flags`` - additional flags passed to ``makechrootpkg`` command, space separated list of strings, optional. * ``makechrootpkg_flags`` - additional flags passed to ``makechrootpkg`` command, space separated list of strings, optional.
* ``scan_paths`` - paths to be used for implicit dependencies scan, space separated list of strings, optional. If any of those paths is matched against the path, it will be added to the allowed list.
* ``triggers`` - list of ``ahriman.core.triggers.Trigger`` class implementation (e.g. ``ahriman.core.report.ReportTrigger ahriman.core.upload.UploadTrigger``) which will be loaded and run at the end of processing, space separated list of strings, optional. You can also specify triggers by their paths, e.g. ``/usr/lib/python3.10/site-packages/ahriman/core/report/report.py.ReportTrigger``. Triggers are run in the order of definition. * ``triggers`` - list of ``ahriman.core.triggers.Trigger`` class implementation (e.g. ``ahriman.core.report.ReportTrigger ahriman.core.upload.UploadTrigger``) which will be loaded and run at the end of processing, space separated list of strings, optional. You can also specify triggers by their paths, e.g. ``/usr/lib/python3.10/site-packages/ahriman/core/report/report.py.ReportTrigger``. Triggers are run in the order of definition.
* ``triggers_known`` - optional list of ``ahriman.core.triggers.Trigger`` class implementations which are not run automatically and used only for trigger discovery and configuration validation. * ``triggers_known`` - optional list of ``ahriman.core.triggers.Trigger`` class implementations which are not run automatically and used only for trigger discovery and configuration validation.
* ``vcs_allowed_age`` - maximal age in seconds of the VCS packages before their version will be updated with its remote source, integer, optional, default is 7 days. * ``vcs_allowed_age`` - maximal age in seconds of the VCS packages before their version will be updated with its remote source, integer, optional, default is 7 days.

View File

@ -379,7 +379,7 @@ After the success build the application extracts all linked libraries and used d
In order to disable this check completely, the ``--no-check-files`` flag can be used. In order to disable this check completely, the ``--no-check-files`` flag can be used.
In addition, there is possibility to control paths which will be used for checking, by using options ``build.allowed_scan_paths`` and ``build.blacklisted_scan_paths``. Leaving ``build.allowed_scan_paths`` blank will effectively disable any check too. In addition, there is possibility to control paths which will be used for checking, by using option ``build.scan_paths``, which supports regular expressions. Leaving ``build.scan_paths`` blank will effectively disable any check too.
How to install built packages How to install built packages
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

View File

@ -50,12 +50,8 @@ allow_read_only = yes
;salt = ;salt =
[build] [build]
; List of paths to be used for implicit dependency scan
allowed_scan_paths = /usr/lib
; List of additional flags passed to archbuild command. ; List of additional flags passed to archbuild command.
;archbuild_flags = ;archbuild_flags =
; List of paths to be excluded for implicit dependency scan. Usually they should be subpaths of allowed_scan_paths
blacklisted_scan_paths = /usr/lib/cmake
; Path to build command ; Path to build command
;build_command = ;build_command =
; List of packages to be ignored during automatic updates. ; List of packages to be ignored during automatic updates.
@ -66,6 +62,8 @@ blacklisted_scan_paths = /usr/lib/cmake
;makechrootpkg_flags = ;makechrootpkg_flags =
; List of additional flags passed to makepkg command. ; List of additional flags passed to makepkg command.
makepkg_flags = --nocolor --ignorearch makepkg_flags = --nocolor --ignorearch
; List of paths to be used for implicit dependency scan. Regular expressions are supported
scan_paths = ^usr/lib(?!/cmake).*$
; List of enabled triggers in the order of calls. ; List of enabled triggers in the order of calls.
triggers = ahriman.core.gitremote.RemotePullTrigger ahriman.core.report.ReportTrigger ahriman.core.upload.UploadTrigger ahriman.core.gitremote.RemotePushTrigger triggers = ahriman.core.gitremote.RemotePullTrigger ahriman.core.report.ReportTrigger ahriman.core.upload.UploadTrigger ahriman.core.gitremote.RemotePushTrigger
; List of well-known triggers. Used only for configuration purposes. ; List of well-known triggers. Used only for configuration purposes.

View File

@ -17,7 +17,6 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
# #
from dataclasses import dataclass
from elftools.elf.dynamic import DynamicSection from elftools.elf.dynamic import DynamicSection
from elftools.elf.elffile import ELFFile from elftools.elf.elffile import ELFFile
from pathlib import Path from pathlib import Path
@ -33,7 +32,6 @@ from ahriman.models.package import Package
from ahriman.models.scan_paths import ScanPaths from ahriman.models.scan_paths import ScanPaths
@dataclass
class PackageArchive: class PackageArchive:
""" """
helper for package archives helper for package archives
@ -45,10 +43,20 @@ class PackageArchive:
scan_paths(ScanPaths): scan paths holder scan_paths(ScanPaths): scan paths holder
""" """
root: Path def __init__(self, root: Path, package: Package, pacman: Pacman, scan_paths: ScanPaths) -> None:
package: Package """
pacman: Pacman default constructor
scan_paths: ScanPaths
Args:
root(Path): path to root filesystem
package(Package): package descriptor
pacman(Pacman): alpm wrapper instance
scan_paths(ScanPaths): scan paths holder
"""
self.root = root
self.package = package
self.pacman = pacman
self.scan_paths = scan_paths
@staticmethod @staticmethod
def dynamic_needed(binary_path: Path) -> list[str]: def dynamic_needed(binary_path: Path) -> list[str]:

View File

@ -169,14 +169,6 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"build": { "build": {
"type": "dict", "type": "dict",
"schema": { "schema": {
"allowed_scan_paths": {
"type": "list",
"coerce": "list",
"schema": {
"type": "path",
"coerce": "absolute_path",
},
},
"archbuild_flags": { "archbuild_flags": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",
@ -185,14 +177,6 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"empty": False, "empty": False,
}, },
}, },
"blacklisted_scan_paths": {
"type": "list",
"coerce": "list",
"schema": {
"type": "path",
"coerce": "absolute_path",
},
},
"build_command": { "build_command": {
"type": "string", "type": "string",
"required": True, "required": True,
@ -226,6 +210,14 @@ CONFIGURATION_SCHEMA: ConfigurationSchema = {
"empty": False, "empty": False,
}, },
}, },
"scan_paths": {
"type": "list",
"coerce": "list",
"schema": {
"type": "string",
"empty": False,
},
},
"triggers": { "triggers": {
"type": "list", "type": "list",
"coerce": "list", "coerce": "list",

View File

@ -23,13 +23,13 @@ from collections.abc import Iterable
from pathlib import Path from pathlib import Path
from tempfile import TemporaryDirectory from tempfile import TemporaryDirectory
from ahriman.core.build_tools.package_archive import PackageArchive
from ahriman.core.build_tools.task import Task from ahriman.core.build_tools.task import Task
from ahriman.core.repository.cleaner import Cleaner from ahriman.core.repository.cleaner import Cleaner
from ahriman.core.repository.package_info import PackageInfo from ahriman.core.repository.package_info import PackageInfo
from ahriman.core.utils import safe_filename from ahriman.core.utils import safe_filename
from ahriman.models.changes import Changes from ahriman.models.changes import Changes
from ahriman.models.package import Package from ahriman.models.package import Package
from ahriman.models.package_archive import PackageArchive
from ahriman.models.package_description import PackageDescription from ahriman.models.package_description import PackageDescription
from ahriman.models.packagers import Packagers from ahriman.models.packagers import Packagers
from ahriman.models.result import Result from ahriman.models.result import Result

View File

@ -80,10 +80,7 @@ class RepositoryProperties(LazyLogging):
self.reporter = Client.load(repository_id, configuration, database, report=report) self.reporter = Client.load(repository_id, configuration, database, report=report)
self.triggers = TriggerLoader.load(repository_id, configuration) self.triggers = TriggerLoader.load(repository_id, configuration)
self.scan_paths = ScanPaths( self.scan_paths = ScanPaths(configuration.getlist("build", "scan_paths", fallback=[]))
allowed_paths=configuration.getpathlist("build", "allowed_scan_paths", fallback=[]),
blacklisted_paths=configuration.getpathlist("build", "blacklisted_scan_paths", fallback=[]),
)
@property @property
def architecture(self) -> str: def architecture(self) -> str:

View File

@ -17,29 +17,33 @@
# You should have received a copy of the GNU General Public License # You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
# #
import re
from dataclasses import dataclass from dataclasses import dataclass
from functools import cached_property
from pathlib import Path from pathlib import Path
@dataclass(frozen=True, kw_only=True) @dataclass(frozen=True)
class ScanPaths: class ScanPaths:
""" """
paths used for scan filesystem paths used for scan filesystem
Attributes: Attributes:
allowed_paths(list[Path]): list of whitelisted paths paths(list[str]): list of regular expressions to be used to match paths
blacklisted_paths(list[Path]): list of paths to be skipped from scan
""" """
allowed_paths: list[Path] paths: list[str]
blacklisted_paths: list[Path]
def __post_init__(self) -> None: @cached_property
def patterns(self) -> list[re.Pattern[str]]:
""" """
compute relative to / paths compiled regular expressions
Returns:
list[re.Pattern]: a list of compiled regular expressions
""" """
object.__setattr__(self, "allowed_paths", [path.relative_to("/") for path in self.allowed_paths]) return [re.compile(path) for path in self.paths]
object.__setattr__(self, "blacklisted_paths", [path.relative_to("/") for path in self.blacklisted_paths])
def is_allowed(self, path: Path) -> bool: def is_allowed(self, path: Path) -> bool:
""" """
@ -49,10 +53,7 @@ class ScanPaths:
path(Path): path to be checked path(Path): path to be checked
Returns: Returns:
bool: ``True`` in case if :attr:`allowed_paths` contains element which is parent for the path and bool: ``True`` in case if :attr:`paths` contains at least one element to which the path is matched
:attr:`blacklisted_paths` doesn't and ``False`` otherwise and ``False`` otherwise
""" """
if any(path.is_relative_to(blacklisted) for blacklisted in self.blacklisted_paths): return any(pattern.match(str(path)) for pattern in self.patterns)
return False # path is blacklisted
# check if we actually have to check this path
return any(path.is_relative_to(allowed) for allowed in self.allowed_paths)

View File

@ -25,6 +25,7 @@ from ahriman.models.remote_source import RemoteSource
from ahriman.models.repository_id import RepositoryId from ahriman.models.repository_id import RepositoryId
from ahriman.models.repository_paths import RepositoryPaths from ahriman.models.repository_paths import RepositoryPaths
from ahriman.models.result import Result from ahriman.models.result import Result
from ahriman.models.scan_paths import ScanPaths
from ahriman.models.user import User from ahriman.models.user import User
from ahriman.models.user_access import UserAccess from ahriman.models.user_access import UserAccess
@ -587,6 +588,20 @@ def result(package_ahriman: Package) -> Result:
return result return result
@pytest.fixture
def scan_paths(configuration: Configuration) -> ScanPaths:
"""
scan paths fixture
Args:
configuration(Configuration): configuration test instance
Returns:
ScanPaths: scan paths test instance
"""
return ScanPaths(configuration.getlist("build", "scan_paths", fallback=[]))
@pytest.fixture @pytest.fixture
def spawner(configuration: Configuration) -> Spawn: def spawner(configuration: Configuration) -> Spawn:
""" """

View File

@ -1,6 +1,35 @@
import pytest import pytest
from pytest_mock import MockerFixture
from typing import Any
from ahriman.core.alpm.pacman import Pacman
from ahriman.core.build_tools.package_archive import PackageArchive
from ahriman.core.build_tools.sources import Sources from ahriman.core.build_tools.sources import Sources
from ahriman.models.package import Package
from ahriman.models.repository_paths import RepositoryPaths
from ahriman.models.scan_paths import ScanPaths
@pytest.fixture
def package_archive_ahriman(package_ahriman: Package, repository_paths: RepositoryPaths, pacman: Pacman,
scan_paths: ScanPaths, passwd: Any, mocker: MockerFixture) -> PackageArchive:
"""
package archive fixture
Args:
package_ahriman(Package): package test instance
repository_paths(RepositoryPaths): repository paths test instance
pacman(Pacman): pacman test instance
scan_paths(ScanPaths): scan paths test instance
passwd(Any): passwd structure test instance
mocker(MockerFixture): mocker object
Returns:
PackageArchive: package archive test instance
"""
mocker.patch("ahriman.models.repository_paths.getpwuid", return_value=passwd)
return PackageArchive(repository_paths.build_directory, package_ahriman, pacman, scan_paths)
@pytest.fixture @pytest.fixture

View File

@ -3,16 +3,16 @@ from pathlib import Path
from pytest_mock import MockerFixture from pytest_mock import MockerFixture
from unittest.mock import MagicMock, PropertyMock from unittest.mock import MagicMock, PropertyMock
from ahriman.core.build_tools.package_archive import PackageArchive
from ahriman.core.exceptions import UnknownPackageError from ahriman.core.exceptions import UnknownPackageError
from ahriman.models.filesystem_package import FilesystemPackage from ahriman.models.filesystem_package import FilesystemPackage
from ahriman.models.package_archive import PackageArchive
def test_dynamic_needed(mocker: MockerFixture) -> None: def test_dynamic_needed(mocker: MockerFixture) -> None:
""" """
must correctly define list of dynamically linked libraries must correctly define list of dynamically linked libraries
""" """
mocker.patch("ahriman.models.package_archive.PackageArchive.is_elf", return_value=True) mocker.patch("ahriman.core.build_tools.package_archive.PackageArchive.is_elf", return_value=True)
linked = PackageArchive.dynamic_needed(Path(".tox") / "tests" / "bin" / "python") linked = PackageArchive.dynamic_needed(Path(".tox") / "tests" / "bin" / "python")
assert linked assert linked
@ -24,7 +24,7 @@ def test_dynamic_needed_not_elf(mocker: MockerFixture) -> None:
""" """
must skip checking if not an elf file must skip checking if not an elf file
""" """
mocker.patch("ahriman.models.package_archive.PackageArchive.is_elf", return_value=False) mocker.patch("ahriman.core.build_tools.package_archive.PackageArchive.is_elf", return_value=False)
assert not PackageArchive.dynamic_needed(Path(".tox") / "tests" / "bin" / "python") assert not PackageArchive.dynamic_needed(Path(".tox") / "tests" / "bin" / "python")
@ -32,7 +32,7 @@ def test_dynamic_needed_no_section(mocker: MockerFixture) -> None:
""" """
must skip checking if there was no dynamic section found must skip checking if there was no dynamic section found
""" """
mocker.patch("ahriman.models.package_archive.PackageArchive.is_elf", return_value=True) mocker.patch("ahriman.core.build_tools.package_archive.PackageArchive.is_elf", return_value=True)
mocker.patch("elftools.elf.elffile.ELFFile.iter_sections", return_value=[]) mocker.patch("elftools.elf.elffile.ELFFile.iter_sections", return_value=[])
assert not PackageArchive.dynamic_needed(Path(".tox") / "tests" / "bin" / "python") assert not PackageArchive.dynamic_needed(Path(".tox") / "tests" / "bin" / "python")
@ -109,8 +109,8 @@ def test_raw_dependencies_packages(package_archive_ahriman: PackageArchive, mock
files=[Path("package2") / "file4", Path("package2") / "file3"], files=[Path("package2") / "file4", Path("package2") / "file3"],
), ),
} }
mocker.patch("ahriman.models.package_archive.PackageArchive.installed_packages", return_value=packages) mocker.patch("ahriman.core.build_tools.package_archive.PackageArchive.installed_packages", return_value=packages)
mocker.patch("ahriman.models.package_archive.PackageArchive.depends_on_paths", return_value=( mocker.patch("ahriman.core.build_tools.package_archive.PackageArchive.depends_on_paths", return_value=(
{"file1", "file3"}, {"file1", "file3"},
{Path("usr") / "dir2", Path("dir3"), Path("package2") / "dir4"}, {Path("usr") / "dir2", Path("dir3"), Path("package2") / "dir4"},
)) ))
@ -165,9 +165,10 @@ def test_depends_on(package_archive_ahriman: PackageArchive, mocker: MockerFixtu
""" """
must extract packages and files which are dependencies for the package must extract packages and files which are dependencies for the package
""" """
raw_mock = mocker.patch("ahriman.models.package_archive.PackageArchive._raw_dependencies_packages", raw_mock = mocker.patch("ahriman.core.build_tools.package_archive.PackageArchive._raw_dependencies_packages",
return_value="1") return_value="1")
refined_mock = mocker.patch("ahriman.models.package_archive.PackageArchive._refine_dependencies", return_value={ refined_mock = mocker.patch(
"ahriman.core.build_tools.package_archive.PackageArchive._refine_dependencies", return_value={
Path("package1") / "file1": [FilesystemPackage(package_name="package1", depends=set(), opt_depends=set())], Path("package1") / "file1": [FilesystemPackage(package_name="package1", depends=set(), opt_depends=set())],
Path("package2") / "file3": [FilesystemPackage(package_name="package2", depends=set(), opt_depends=set())], Path("package2") / "file3": [FilesystemPackage(package_name="package2", depends=set(), opt_depends=set())],
Path("package2") / "dir4": [FilesystemPackage(package_name="package2", depends=set(), opt_depends=set())], Path("package2") / "dir4": [FilesystemPackage(package_name="package2", depends=set(), opt_depends=set())],
@ -175,7 +176,8 @@ def test_depends_on(package_archive_ahriman: PackageArchive, mocker: MockerFixtu
FilesystemPackage(package_name="package1", depends=set(), opt_depends=set()), FilesystemPackage(package_name="package1", depends=set(), opt_depends=set()),
FilesystemPackage(package_name="package2", depends=set(), opt_depends=set()), FilesystemPackage(package_name="package2", depends=set(), opt_depends=set()),
], ],
}) }
)
result = package_archive_ahriman.depends_on() result = package_archive_ahriman.depends_on()
raw_mock.assert_called_once_with() raw_mock.assert_called_once_with()
@ -194,8 +196,9 @@ def test_depends_on_paths(package_archive_ahriman: PackageArchive, mocker: Mocke
""" """
package_dir = package_archive_ahriman.root / "build" / \ package_dir = package_archive_ahriman.root / "build" / \
package_archive_ahriman.package.base / "pkg" / package_archive_ahriman.package.base package_archive_ahriman.package.base / "pkg" / package_archive_ahriman.package.base
dynamic_mock = mocker.patch("ahriman.models.package_archive.PackageArchive.dynamic_needed", return_value=["lib"]) dynamic_mock = mocker.patch("ahriman.core.build_tools.package_archive.PackageArchive.dynamic_needed",
walk_mock = mocker.patch("ahriman.models.package_archive.walk", return_value=[ return_value=["lib"])
walk_mock = mocker.patch("ahriman.core.build_tools.package_archive.walk", return_value=[
package_dir / "root" / "file", package_dir / "root" / "file",
Path("directory"), Path("directory"),
]) ])
@ -213,7 +216,7 @@ def test_installed_packages(package_archive_ahriman: PackageArchive, mocker: Moc
""" """
must load list of installed packages and their files must load list of installed packages and their files
""" """
walk_mock = mocker.patch("ahriman.models.package_archive.walk", return_value=[ walk_mock = mocker.patch("ahriman.core.build_tools.package_archive.walk", return_value=[
Path("ahriman-2.13.3-1") / "desc", Path("ahriman-2.13.3-1") / "desc",
Path("ahriman-2.13.3-1") / "files", Path("ahriman-2.13.3-1") / "files",
]) ])

View File

@ -24,7 +24,7 @@ def test_process_build(executor: Executor, package_ahriman: Package, passwd: Any
move_mock = mocker.patch("shutil.move") move_mock = mocker.patch("shutil.move")
status_client_mock = mocker.patch("ahriman.core.status.Client.set_building") status_client_mock = mocker.patch("ahriman.core.status.Client.set_building")
commit_sha_mock = mocker.patch("ahriman.core.status.local_client.LocalClient.package_changes_update") commit_sha_mock = mocker.patch("ahriman.core.status.local_client.LocalClient.package_changes_update")
depends_on_mock = mocker.patch("ahriman.models.package_archive.PackageArchive.depends_on", depends_on_mock = mocker.patch("ahriman.core.build_tools.package_archive.PackageArchive.depends_on",
return_value=Dependencies()) return_value=Dependencies())
dependencies_mock = mocker.patch("ahriman.core.status.local_client.LocalClient.package_dependencies_update") dependencies_mock = mocker.patch("ahriman.core.status.local_client.LocalClient.package_dependencies_update")

View File

@ -1,24 +1,17 @@
import pytest import pytest
from typing import Any
from unittest.mock import MagicMock, PropertyMock from unittest.mock import MagicMock, PropertyMock
from pytest_mock import MockerFixture
from ahriman import __version__ from ahriman import __version__
from ahriman.core.alpm.pacman import Pacman
from ahriman.core.alpm.remote import AUR from ahriman.core.alpm.remote import AUR
from ahriman.core.configuration import Configuration
from ahriman.models.build_status import BuildStatus, BuildStatusEnum from ahriman.models.build_status import BuildStatus, BuildStatusEnum
from ahriman.models.counters import Counters from ahriman.models.counters import Counters
from ahriman.models.filesystem_package import FilesystemPackage from ahriman.models.filesystem_package import FilesystemPackage
from ahriman.models.internal_status import InternalStatus from ahriman.models.internal_status import InternalStatus
from ahriman.models.package import Package from ahriman.models.package import Package
from ahriman.models.package_archive import PackageArchive
from ahriman.models.package_description import PackageDescription from ahriman.models.package_description import PackageDescription
from ahriman.models.package_source import PackageSource from ahriman.models.package_source import PackageSource
from ahriman.models.remote_source import RemoteSource from ahriman.models.remote_source import RemoteSource
from ahriman.models.repository_paths import RepositoryPaths
from ahriman.models.scan_paths import ScanPaths
@pytest.fixture @pytest.fixture
@ -77,27 +70,6 @@ def internal_status(counters: Counters) -> InternalStatus:
repository="aur-clone") repository="aur-clone")
@pytest.fixture
def package_archive_ahriman(package_ahriman: Package, repository_paths: RepositoryPaths, pacman: Pacman,
scan_paths: ScanPaths, passwd: Any, mocker: MockerFixture) -> PackageArchive:
"""
package archive fixture
Args:
package_ahriman(Package): package test instance
repository_paths(RepositoryPaths): repository paths test instance
pacman(Pacman): pacman test instance
scan_paths(ScanPaths): scan paths test instance
passwd(Any): passwd structure test instance
mocker(MockerFixture): mocker object
Returns:
PackageArchive: package archive test instance
"""
mocker.patch("ahriman.models.repository_paths.getpwuid", return_value=passwd)
return PackageArchive(repository_paths.build_directory, package_ahriman, pacman, scan_paths)
@pytest.fixture @pytest.fixture
def package_tpacpi_bat_git() -> Package: def package_tpacpi_bat_git() -> Package:
""" """
@ -161,20 +133,3 @@ def pyalpm_package_description_ahriman(package_description_ahriman: PackageDescr
type(mock).provides = PropertyMock(return_value=package_description_ahriman.provides) type(mock).provides = PropertyMock(return_value=package_description_ahriman.provides)
type(mock).url = PropertyMock(return_value=package_description_ahriman.url) type(mock).url = PropertyMock(return_value=package_description_ahriman.url)
return mock return mock
@pytest.fixture
def scan_paths(configuration: Configuration) -> ScanPaths:
"""
scan paths fixture
Args:
configuration(Configuration): configuration test instance
Returns:
ScanPaths: scan paths test instance
"""
return ScanPaths(
allowed_paths=configuration.getpathlist("build", "allowed_scan_paths"),
blacklisted_paths=configuration.getpathlist("build", "blacklisted_scan_paths"),
)

View File

@ -3,40 +3,30 @@ from pathlib import Path
from ahriman.models.scan_paths import ScanPaths from ahriman.models.scan_paths import ScanPaths
def test_post_init(scan_paths: ScanPaths) -> None:
"""
must convert paths to / relative
"""
assert all(not path.is_absolute() for path in scan_paths.allowed_paths)
assert all(not path.is_absolute() for path in scan_paths.blacklisted_paths)
def test_is_allowed() -> None: def test_is_allowed() -> None:
""" """
must check if path is subpath of one in allowed list must check if path is subpath of one in allowed list
""" """
assert ScanPaths(allowed_paths=[Path("/") / "usr"], blacklisted_paths=[]).is_allowed(Path("usr")) assert ScanPaths(["usr"]).is_allowed(Path("usr"))
assert ScanPaths(allowed_paths=[Path("/") / "usr"], blacklisted_paths=[]).is_allowed(Path("usr") / "lib") assert ScanPaths(["usr"]).is_allowed(Path("usr") / "lib")
assert not ScanPaths(allowed_paths=[Path("/") / "usr"], blacklisted_paths=[]).is_allowed(Path("var")) assert not ScanPaths(["usr"]).is_allowed(Path("var"))
assert ScanPaths(["usr(?!/lib)"]).is_allowed(Path("usr"))
assert ScanPaths(["usr(?!/lib)", "var"]).is_allowed(Path("var"))
assert not ScanPaths(["usr(?!/lib)"]).is_allowed(Path("usr") / "lib")
def test_is_blacklisted() -> None: def test_is_allowed_default(scan_paths: ScanPaths) -> None:
""" """
must check if path is not subpath of one in blacklist must provide expected default configuration
""" """
assert ScanPaths( assert not scan_paths.is_allowed(Path("usr"))
allowed_paths=[Path("/") / "usr"], assert not scan_paths.is_allowed(Path("var"))
blacklisted_paths=[Path("/") / "usr" / "lib"],
).is_allowed(Path("usr")) assert scan_paths.is_allowed(Path("usr") / "lib")
assert ScanPaths( assert scan_paths.is_allowed(Path("usr") / "lib" / "libm.so")
allowed_paths=[Path("/") / "usr", Path("/") / "var"],
blacklisted_paths=[Path("/") / "usr" / "lib"], # cmake case
).is_allowed(Path("var")) assert scan_paths.is_allowed(Path("usr") / "lib" / "libcmake.so")
assert not ScanPaths( assert not scan_paths.is_allowed(Path("usr") / "lib" / "cmake")
allowed_paths=[Path("/") / "usr"], assert not scan_paths.is_allowed(Path("usr") / "lib" / "cmake" / "file.cmake")
blacklisted_paths=[Path("/") / "usr" / "lib"],
).is_allowed(Path(" usr") / "lib")
assert not ScanPaths(
allowed_paths=[Path("/") / "usr"],
blacklisted_paths=[Path("/") / "usr" / "lib"],
).is_allowed(Path("usr") / "lib" / "qt")

View File

@ -20,13 +20,12 @@ salt = salt
allow_read_only = no allow_read_only = no
[build] [build]
allowed_scan_paths = /usr/lib
archbuild_flags = archbuild_flags =
blacklisted_scan_paths = /usr/lib/cmake
build_command = extra-x86_64-build build_command = extra-x86_64-build
ignore_packages = ignore_packages =
makechrootpkg_flags = makechrootpkg_flags =
makepkg_flags = --skippgpcheck makepkg_flags = --skippgpcheck
scan_paths = ^usr/lib(?!/cmake).*$
triggers = ahriman.core.report.ReportTrigger ahriman.core.upload.UploadTrigger triggers = ahriman.core.report.ReportTrigger ahriman.core.upload.UploadTrigger
triggers_known = ahriman.core.distributed.WorkerLoaderTrigger ahriman.core.distributed.WorkerRegisterTrigger ahriman.core.distributed.WorkerTrigger ahriman.core.distributed.WorkerUnregisterTrigger ahriman.core.gitremote.RemotePullTrigger ahriman.core.gitremote.RemotePushTrigger ahriman.core.report.ReportTrigger ahriman.core.upload.UploadTrigger ahriman.core.support.KeyringTrigger ahriman.core.support.MirrorlistTrigger triggers_known = ahriman.core.distributed.WorkerLoaderTrigger ahriman.core.distributed.WorkerRegisterTrigger ahriman.core.distributed.WorkerTrigger ahriman.core.distributed.WorkerUnregisterTrigger ahriman.core.gitremote.RemotePullTrigger ahriman.core.gitremote.RemotePushTrigger ahriman.core.report.ReportTrigger ahriman.core.upload.UploadTrigger ahriman.core.support.KeyringTrigger ahriman.core.support.MirrorlistTrigger