Compare commits

..

1 Commits

Author SHA1 Message Date
3a78b42df6 feat: add blacklisted paths to implicit dependencies processing
It has been found that in some cases additional packages have been added
as dependencies, like usr/share/applications, usr/lib/cmake, etc

This commit adds an ability to blacklist specific paths from processing
2024-08-21 15:26:43 +03:00
32 changed files with 6854 additions and 7391 deletions

View File

@ -82,7 +82,6 @@ limit-inference-results=100
# List of plugins (as comma separated values of python module names) to load,
# usually to register additional checkers.
load-plugins=pylint.extensions.docparams,
pylint.extensions.bad_builtin,
definition_order,
import_order,
@ -132,8 +131,6 @@ attr-naming-style=snake_case
# style.
#attr-rgx=
bad-functions=print,
# Bad variable names which should always be refused, separated by a comma.
bad-names=foo,
bar,

View File

@ -132,7 +132,7 @@ Again, the most checks can be performed by `tox` command, though some additional
* For any path interactions `pathlib.Path` must be used.
* Configuration interactions must go through `ahriman.core.configuration.Configuration` class instance.
* In case if class load requires some actions, it is recommended to create class method which can be used for class instantiating.
* The most (expected) exceptions must be handled and printed to log, allowing service to continue work. However, fatal and (in some cases) unexpected exceptions may lead to the application termination.
* The code must follow the exception safety, unless it is explicitly asked by end user. It means that most exceptions must be handled and printed to log, no other actions must be done (e.g. raising another exception).
* Exceptions without parameters should be raised without parentheses, e.g.:
```python

View File

@ -40,5 +40,3 @@ The application provides reasonable defaults which allow to use it out-of-box; h
* [Build status page](https://ahriman-demo.arcanis.me). You can log in as `demo` user by using `demo` password. However, you will not be able to run tasks. [HTTP API documentation](https://ahriman-demo.arcanis.me/api-docs) is also available.
* [Repository index](https://repo.arcanis.me/arcanisrepo/x86_64/).
* [Telegram feed](https://t.me/arcanisrepo).
Do you have any success story? You can [share it](https://github.com/arcan1s/ahriman/issues/new?template=04-discussion.md)!

File diff suppressed because it is too large Load Diff

Before

Width:  |  Height:  |  Size: 1.2 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

@ -228,14 +228,6 @@ ahriman.models.result module
:no-undoc-members:
:show-inheritance:
ahriman.models.scan\_paths module
---------------------------------
.. automodule:: ahriman.models.scan_paths
:members:
:no-undoc-members:
:show-inheritance:
ahriman.models.sign\_settings module
------------------------------------

View File

@ -370,16 +370,7 @@ TL;DR
You can even rebuild the whole repository (which is particular useful in case if you would like to change packager) if you do not supply ``--depends-on`` option. This action will automatically increment ``pkgrel`` value; in case if you don't want to, the ``--no-increment`` option has to be supplied.
However, note that you do not need to rebuild repository in case if you just changed signing option, just use ``repo-sign`` command instead.
Automated broken dependencies detection
"""""""""""""""""""""""""""""""""""""""
After the success build the application extracts all linked libraries and used directories and stores them in database. During the check process, the application extracts pacman databases and checks if file names have been changed (e.g. new python release caused ``/usr/lib/python3.x`` directory renaming to ``/usr/lib/python3.y`` or soname for a linked library has been changed). In case if broken dependencies have been detected, the package will be added to the rebuild queue.
In order to disable this check completely, the ``--no-check-files`` flag can be used.
In addition, there is possibility to control paths which will be used for checking, by using options ``build.allowed_scan_paths`` and ``build.blacklisted_scan_paths``. Leaving ``build.allowed_scan_paths`` blank will effectively disable any check too.
However, note that you do not need to rebuild repository in case if you just changed signing option, just use ``repo-sign`` command instead.
How to install built packages
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

View File

@ -1,7 +1,7 @@
# Maintainer: Evgeniy Alekseev
pkgname='ahriman'
pkgver=2.14.1
pkgver=2.13.8
pkgrel=1
pkgdesc="ArcH linux ReposItory MANager"
arch=('any')

View File

@ -1,4 +1,4 @@
.TH AHRIMAN "1" "2024\-09\-04" "ahriman" "Generated Python Manual"
.TH AHRIMAN "1" "2024\-05\-12" "ahriman" "Generated Python Manual"
.SH NAME
ahriman
.SH SYNOPSIS
@ -391,7 +391,7 @@ PKGBUILD variable or function name. If variable is a function, it must end with
path to file which contains function or variable value. If not set, the value will be read from stdin
.SH COMMAND \fI\,'ahriman patch\-list'\/\fR
usage: ahriman patch\-list [\-h] [\-e] [\-v VARIABLE] package
usage: ahriman patch\-list [\-h] [\-e] [\-v VARIABLE] [package]
list available patches for the package

View File

@ -86,7 +86,7 @@ _shtab_ahriman_options=(
{-a,--architecture}"[filter by target architecture (default\: None)]:architecture:"
{-c,--configuration}"[configuration path (default\: \/etc\/ahriman.ini)]:configuration:"
"--force[force run, remove file lock (default\: False)]"
{-l,--lock}"[lock file (default\: ahriman.pid)]:lock:"
{-l,--lock}"[lock file (default\: \/tmp\/ahriman.lock)]:lock:"
"--log-handler[explicit log handler specification. If none set, the handler will be guessed from environment (default\: None)]:log_handler:(console syslog journald)"
{-q,--quiet}"[force disable any logging (default\: False)]"
{--report,--no-report}"[force enable or disable reporting to web service (default\: True)]:report:"
@ -280,7 +280,7 @@ _shtab_ahriman_patch_list_options=(
"(- : *)"{-h,--help}"[show this help message and exit]"
{-e,--exit-code}"[return non-zero exit status if result is empty (default\: False)]"
"*"{-v,--variable}"[if set, show only patches for specified PKGBUILD variables (default\: None)]:variable:"
":package base:"
":package base (default\: None):"
)
_shtab_ahriman_patch_remove_options=(

View File

@ -17,4 +17,4 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
__version__ = "2.14.1"
__version__ = "2.13.8"

View File

@ -98,7 +98,6 @@ class Patch(Handler):
PkgbuildPatch: created patch for the PKGBUILD function
"""
if patch_path is None:
# pylint: disable=bad-builtin
print("Post new function or variable value below. Press Ctrl-D to finish:", file=sys.stderr)
patch = "".join(list(sys.stdin))
else:

View File

@ -77,5 +77,5 @@ class Update(Handler):
Callable[[str], None]: in case if dry_run is set it will return print, logger otherwise
"""
def inner(line: str) -> None:
return print(line) if dry_run else application.logger.info(line) # pylint: disable=bad-builtin
return print(line) if dry_run else application.logger.info(line)
return inner

View File

@ -75,9 +75,7 @@ class Lock(LazyLogging):
"""
self.path: Path | None = None
if args.lock is not None:
self.path = args.lock
if not repository_id.is_empty:
self.path = self.path.with_stem(f"{args.lock.stem}_{repository_id.id}")
self.path = args.lock.with_stem(f"{args.lock.stem}_{repository_id.id}")
if not self.path.is_absolute():
# prepend full path to the lock file
self.path = Path("/") / "run" / "ahriman" / self.path

View File

@ -43,7 +43,7 @@ class Pacman(LazyLogging):
configuration(Configuration): configuration instance
refresh_database(PacmanSynchronization): synchronize local cache to remote
repository_id(RepositoryId): repository unique identifier
repository_paths(RepositoryPaths): repository paths instance
repository_path(RepositoryPaths): repository paths instance
"""
def __init__(self, repository_id: RepositoryId, configuration: Configuration, *,
@ -188,8 +188,8 @@ class Pacman(LazyLogging):
Returns:
dict[str, set[str]]: map of package name to its list of files
"""
def extract(tar: tarfile.TarFile, versions: dict[str, str]) -> Generator[tuple[str, set[str]], None, None]:
for package_name, version in versions.items():
def extract(tar: tarfile.TarFile, package_names: dict[str, str]) -> Generator[tuple[str, set[str]], None, None]:
for package_name, version in package_names.items():
path = Path(f"{package_name}-{version}") / "files"
try:
content = tar.extractfile(str(path))

View File

@ -59,8 +59,7 @@ class PacmanDatabase(SyncHttpClient):
self.sync_files_database = configuration.getboolean("alpm", "sync_files_database")
@staticmethod
def copy(remote_path: Path, local_path: Path) -> None:
def copy(self, remote_path: Path, local_path: Path) -> None:
"""
copy local database file

View File

@ -19,7 +19,6 @@
#
import shutil
from collections.abc import Generator
from pathlib import Path
from ahriman.core.exceptions import CalledProcessError
@ -39,14 +38,10 @@ class Sources(LazyLogging):
DEFAULT_BRANCH(str): (class attribute) default branch to process git repositories.
Must be used only for local stored repositories, use RemoteSource descriptor instead for real packages
DEFAULT_COMMIT_AUTHOR(tuple[str, str]): (class attribute) default commit author to be used if none set
GITCONFIG(dict[str, str]): (class attribute) git config options to suppress annoying hints
"""
DEFAULT_BRANCH = "master" # default fallback branch
DEFAULT_COMMIT_AUTHOR = ("ahriman", "ahriman@localhost")
GITCONFIG = {
"init.defaultBranch": DEFAULT_BRANCH,
}
@staticmethod
def changes(source_dir: Path, last_commit_sha: str | None) -> str | None:
@ -111,15 +106,15 @@ class Sources(LazyLogging):
instance.fetch_until(sources_dir, branch=branch)
elif remote.git_url is not None:
instance.logger.info("clone remote %s to %s using branch %s", remote.git_url, sources_dir, branch)
check_output(*instance.git(), "clone", "--quiet", "--depth", "1", "--branch", branch, "--single-branch",
check_output("git", "clone", "--quiet", "--depth", "1", "--branch", branch, "--single-branch",
remote.git_url, str(sources_dir), cwd=sources_dir.parent, logger=instance.logger)
else:
# it will cause an exception later
instance.logger.error("%s is not initialized, but no remote provided", sources_dir)
# and now force reset to our branch
check_output(*instance.git(), "checkout", "--force", branch, cwd=sources_dir, logger=instance.logger)
check_output(*instance.git(), "reset", "--quiet", "--hard", f"origin/{branch}",
check_output("git", "checkout", "--force", branch, cwd=sources_dir, logger=instance.logger)
check_output("git", "reset", "--quiet", "--hard", f"origin/{branch}",
cwd=sources_dir, logger=instance.logger)
# move content if required
@ -141,7 +136,7 @@ class Sources(LazyLogging):
bool: True in case if there is any remote and false otherwise
"""
instance = Sources()
remotes = check_output(*instance.git(), "remote", cwd=sources_dir, logger=instance.logger)
remotes = check_output("git", "remote", cwd=sources_dir, logger=instance.logger)
return bool(remotes)
@staticmethod
@ -155,7 +150,7 @@ class Sources(LazyLogging):
instance = Sources()
if not (sources_dir / ".git").is_dir():
# skip initializing in case if it was already
check_output(*instance.git(), "init", "--quiet", "--initial-branch", instance.DEFAULT_BRANCH,
check_output("git", "init", "--quiet", "--initial-branch", instance.DEFAULT_BRANCH,
cwd=sources_dir, logger=instance.logger)
# extract local files...
@ -225,7 +220,7 @@ class Sources(LazyLogging):
return # no changes to push, just skip action
git_url, branch = remote.git_source()
check_output(*instance.git(), "push", "--quiet", git_url, branch, cwd=sources_dir, logger=instance.logger)
check_output("git", "push", "--quiet", git_url, branch, cwd=sources_dir, logger=instance.logger)
def add(self, sources_dir: Path, *pattern: str, intent_to_add: bool = False) -> None:
"""
@ -246,7 +241,7 @@ class Sources(LazyLogging):
self.logger.info("found matching files %s", found_files)
# add them to index
args = ["--intent-to-add"] if intent_to_add else []
check_output(*self.git(), "add", *args, *[str(fn.relative_to(sources_dir)) for fn in found_files],
check_output("git", "add", *args, *[str(fn.relative_to(sources_dir)) for fn in found_files],
cwd=sources_dir, logger=self.logger)
def commit(self, sources_dir: Path, message: str | None = None,
@ -269,16 +264,15 @@ class Sources(LazyLogging):
if message is None:
message = f"Autogenerated commit at {utcnow()}"
args = ["--message", message]
environment: dict[str, str] = {}
if commit_author is None:
commit_author = self.DEFAULT_COMMIT_AUTHOR
user, email = commit_author
gitconfig = {
"user.email": email,
"user.name": user,
}
environment["GIT_AUTHOR_NAME"] = environment["GIT_COMMITTER_NAME"] = user
environment["GIT_AUTHOR_EMAIL"] = environment["GIT_COMMITTER_EMAIL"] = email
check_output(*self.git(gitconfig), "commit", "--quiet", *args, cwd=sources_dir, logger=self.logger)
check_output("git", "commit", "--quiet", *args, cwd=sources_dir, logger=self.logger, environment=environment)
return True
@ -296,7 +290,7 @@ class Sources(LazyLogging):
args = []
if sha is not None:
args.append(sha)
return check_output(*self.git(), "diff", *args, cwd=sources_dir, logger=self.logger)
return check_output("git", "diff", *args, cwd=sources_dir, logger=self.logger)
def fetch_until(self, sources_dir: Path, *, branch: str | None = None, commit_sha: str | None = None) -> None:
"""
@ -312,37 +306,18 @@ class Sources(LazyLogging):
commits_count = 1
while commit_sha is not None:
command = self.git() + ["fetch", "--quiet", "--depth", str(commits_count)]
command = ["git", "fetch", "--quiet", "--depth", str(commits_count)]
if branch is not None:
command += ["origin", branch]
check_output(*command, cwd=sources_dir, logger=self.logger) # fetch one more level
try:
# check if there is an object in current git directory
check_output(*self.git(), "cat-file", "-e", commit_sha, cwd=sources_dir, logger=self.logger)
check_output("git", "cat-file", "-e", commit_sha, cwd=sources_dir, logger=self.logger)
commit_sha = None # reset search
except CalledProcessError:
commits_count += 1 # increase depth
def git(self, gitconfig: dict[str, str] | None = None) -> list[str]:
"""
git command prefix
Args:
gitconfig(dict[str, str] | None, optional): additional git config flags if any (Default value = None)
Returns:
list[str]: git command prefix with valid default flags
"""
gitconfig = gitconfig or {}
def configuration_flags() -> Generator[str, None, None]:
for option, value in (self.GITCONFIG | gitconfig).items():
yield "-c"
yield f"{option}=\"{value}\""
return ["git"] + list(configuration_flags())
def has_changes(self, sources_dir: Path) -> bool:
"""
check if there are changes in current git tree
@ -354,7 +329,7 @@ class Sources(LazyLogging):
bool: True if there are uncommitted changes and False otherwise
"""
# there is --exit-code argument to diff, however, there might be other process errors
changes = check_output(*self.git(), "diff", "--cached", "--name-only", cwd=sources_dir, logger=self.logger)
changes = check_output("git", "diff", "--cached", "--name-only", cwd=sources_dir, logger=self.logger)
return bool(changes)
def head(self, sources_dir: Path, ref_name: str = "HEAD") -> str:
@ -369,7 +344,7 @@ class Sources(LazyLogging):
str: HEAD commit hash
"""
# we might want to parse git files instead though
return check_output(*self.git(), "rev-parse", ref_name, cwd=sources_dir, logger=self.logger)
return check_output("git", "rev-parse", ref_name, cwd=sources_dir, logger=self.logger)
def move(self, pkgbuild_dir: Path, sources_dir: Path) -> None:
"""
@ -397,7 +372,7 @@ class Sources(LazyLogging):
# create patch
self.logger.info("apply patch %s from database at %s", patch.key, sources_dir)
if patch.is_plain_diff:
check_output(*self.git(), "apply", "--ignore-space-change", "--ignore-whitespace",
check_output("git", "apply", "--ignore-space-change", "--ignore-whitespace",
cwd=sources_dir, input_data=patch.serialize(), logger=self.logger)
else:
patch.write(sources_dir / "PKGBUILD")

View File

@ -27,7 +27,6 @@ from ahriman.core.configuration import Configuration
from ahriman.core.database.migrations import Migrations
from ahriman.core.database.operations import AuthOperations, BuildOperations, ChangesOperations, \
DependenciesOperations, LogsOperations, PackageOperations, PatchOperations
from ahriman.models.repository_id import RepositoryId
# pylint: disable=too-many-ancestors
@ -103,26 +102,23 @@ class SQLite(
self.with_connection(lambda connection: Migrations.migrate(connection, configuration))
paths.chown(self.path)
def package_clear(self, package_base: str, repository_id: RepositoryId | None = None) -> None:
def package_clear(self, package_base: str) -> None:
"""
completely remove package from all tables
Args:
package_base(str): package base to remove
repository_id(RepositoryId, optional): repository unique identifier override (Default value = None)
Examples:
This method completely removes the package from all tables and must be used, e.g. on package removal::
>>> database.package_clear("ahriman")
"""
self.build_queue_clear(package_base, repository_id)
self.patches_remove(package_base, None)
self.logs_remove(package_base, None, repository_id)
self.changes_remove(package_base, repository_id)
self.dependencies_remove(package_base, repository_id)
self.package_remove(package_base, repository_id)
self.build_queue_clear(package_base)
self.patches_remove(package_base, [])
self.logs_remove(package_base, None)
self.changes_remove(package_base)
self.dependencies_remove(package_base)
# remove local cache too
self._repository_paths.tree_clear(package_base)

View File

@ -310,7 +310,7 @@ class Client:
def set_unknown(self, package: Package) -> None:
"""
set package status to unknown. Unlike other methods, this method also checks if package is known,
and - in case if it is - it silently skips update
and - in case if it is - it silently skips updatd
Args:
package(Package): current package properties

View File

@ -184,7 +184,7 @@ class LocalClient(Client):
Args:
package_base(str): package base to remove
"""
self.database.package_clear(package_base, self.repository_id)
self.database.package_clear(package_base)
def package_status_update(self, package_base: str, status: BuildStatusEnum) -> None:
"""

View File

@ -140,6 +140,7 @@ class Watcher(LazyLogging):
with self._lock:
self._known.pop(package_base, None)
self.client.package_remove(package_base)
self.package_logs_remove(package_base, None)
def package_status_update(self, package_base: str, status: BuildStatusEnum) -> None:
"""

View File

@ -41,12 +41,9 @@ class RepositoryId:
Returns:
str: unique id for this repository
Raises:
ValueError: if repository identifier is empty
"""
if self.is_empty:
raise ValueError("Repository ID is called on empty repository identifier")
return ""
return f"{self.architecture}-{self.name}" # basically the same as used for command line
@property

View File

@ -113,7 +113,7 @@ class RepositoryPaths(LazyLogging):
Returns:
Path: full patch to devtools chroot directory
"""
# for the chroot directory devtools will create own tree, and we don't have to specify architecture here
# for the chroot directory devtools will create own tree, and we don"t have to specify architecture here
return self.root / "chroot" / self.repository_id.name
@property

View File

@ -67,7 +67,7 @@ class WaiterTaskFinished(WaiterResult):
indicates whether the waiter completed with success or not
Returns:
Literal[True]: always ``True``
Literal[True]: always False
"""
return True
@ -82,7 +82,7 @@ class WaiterTimedOut(WaiterResult):
indicates whether the waiter completed with success or not
Returns:
Literal[False]: always ``False``
Literal[False]: always False
"""
return False
@ -108,7 +108,7 @@ class Waiter:
check if timer is out
Returns:
bool: ``True`` in case current monotonic time is more than :attr:`start_time` and :attr:`wait_timeout`
bool: True in case current monotonic time is more than :attr:`start_time` and :attr:`wait_timeout`
doesn't equal to 0
"""
since_start = time.monotonic() - self.start_time
@ -124,7 +124,7 @@ class Waiter:
**kwargs(Params.kwargs): keyword arguments for check call
Returns:
WaiterResult: waiter result object
WaiterResult: consumed time in seconds
"""
while not (timed_out := self.is_timed_out()) and in_progress(*args, **kwargs):
time.sleep(self.interval)

View File

@ -14,7 +14,6 @@ from ahriman.core.configuration import Configuration
from ahriman.core.exceptions import DuplicateRunError, UnsafeRunError
from ahriman.models.build_status import BuildStatus, BuildStatusEnum
from ahriman.models.internal_status import InternalStatus
from ahriman.models.repository_id import RepositoryId
def test_path(args: argparse.Namespace, configuration: Configuration) -> None:
@ -31,8 +30,6 @@ def test_path(args: argparse.Namespace, configuration: Configuration) -> None:
args.lock = Path("ahriman.pid")
assert Lock(args, repository_id, configuration).path == Path("/run/ahriman/ahriman_x86_64-aur-clone.pid")
assert Lock(args, RepositoryId("", ""), configuration).path == Path("/run/ahriman/ahriman.pid")
with pytest.raises(ValueError):
args.lock = Path("/")
assert Lock(args, repository_id, configuration).path # special case

View File

@ -8,12 +8,12 @@ from ahriman.core.alpm.pacman_database import PacmanDatabase
from ahriman.core.exceptions import PacmanError
def test_copy(mocker: MockerFixture) -> None:
def test_copy(pacman_database: PacmanDatabase, mocker: MockerFixture) -> None:
"""
must copy loca database file
"""
copy_mock = mocker.patch("shutil.copy")
PacmanDatabase.copy(Path("remote"), Path("local"))
pacman_database.copy(Path("remote"), Path("local"))
copy_mock.assert_called_once_with(Path("remote"), Path("local"))

View File

@ -74,7 +74,7 @@ def test_fetch_empty(remote_source: RemoteSource, mocker: MockerFixture) -> None
check_output_mock.assert_not_called()
def test_fetch_existing(sources: Sources, remote_source: RemoteSource, mocker: MockerFixture) -> None:
def test_fetch_existing(remote_source: RemoteSource, mocker: MockerFixture) -> None:
"""
must fetch new package via fetch command
"""
@ -86,19 +86,18 @@ def test_fetch_existing(sources: Sources, remote_source: RemoteSource, mocker: M
head_mock = mocker.patch("ahriman.core.build_tools.sources.Sources.head", return_value="sha")
local = Path("local")
assert sources.fetch(local, remote_source) == "sha"
assert Sources.fetch(local, remote_source) == "sha"
fetch_mock.assert_called_once_with(local, branch=remote_source.branch)
check_output_mock.assert_has_calls([
MockCall(*sources.git(), "checkout", "--force", remote_source.branch,
cwd=local, logger=pytest.helpers.anyvar(int)),
MockCall(*sources.git(), "reset", "--quiet", "--hard", f"origin/{remote_source.branch}",
MockCall("git", "checkout", "--force", remote_source.branch, cwd=local, logger=pytest.helpers.anyvar(int)),
MockCall("git", "reset", "--quiet", "--hard", f"origin/{remote_source.branch}",
cwd=local, logger=pytest.helpers.anyvar(int)),
])
move_mock.assert_called_once_with(local.resolve(), local)
head_mock.assert_called_once_with(local)
def test_fetch_new(sources: Sources, remote_source: RemoteSource, mocker: MockerFixture) -> None:
def test_fetch_new(remote_source: RemoteSource, mocker: MockerFixture) -> None:
"""
must fetch new package via clone command
"""
@ -108,21 +107,19 @@ def test_fetch_new(sources: Sources, remote_source: RemoteSource, mocker: Mocker
head_mock = mocker.patch("ahriman.core.build_tools.sources.Sources.head", return_value="sha")
local = Path("local")
assert sources.fetch(local, remote_source) == "sha"
assert Sources.fetch(local, remote_source) == "sha"
check_output_mock.assert_has_calls([
MockCall(*sources.git(), "clone", "--quiet", "--depth", "1", "--branch", remote_source.branch,
"--single-branch", remote_source.git_url, str(local),
cwd=local.parent, logger=pytest.helpers.anyvar(int)),
MockCall(*sources.git(), "checkout", "--force", remote_source.branch,
cwd=local, logger=pytest.helpers.anyvar(int)),
MockCall(*sources.git(), "reset", "--quiet", "--hard", f"origin/{remote_source.branch}",
MockCall("git", "clone", "--quiet", "--depth", "1", "--branch", remote_source.branch, "--single-branch",
remote_source.git_url, str(local), cwd=local.parent, logger=pytest.helpers.anyvar(int)),
MockCall("git", "checkout", "--force", remote_source.branch, cwd=local, logger=pytest.helpers.anyvar(int)),
MockCall("git", "reset", "--quiet", "--hard", f"origin/{remote_source.branch}",
cwd=local, logger=pytest.helpers.anyvar(int))
])
move_mock.assert_called_once_with(local.resolve(), local)
head_mock.assert_called_once_with(local)
def test_fetch_new_without_remote(sources: Sources, mocker: MockerFixture) -> None:
def test_fetch_new_without_remote(mocker: MockerFixture) -> None:
"""
must fetch nothing in case if no remote set
"""
@ -132,11 +129,10 @@ def test_fetch_new_without_remote(sources: Sources, mocker: MockerFixture) -> No
head_mock = mocker.patch("ahriman.core.build_tools.sources.Sources.head", return_value="sha")
local = Path("local")
assert sources.fetch(local, RemoteSource(source=PackageSource.Archive)) == "sha"
assert Sources.fetch(local, RemoteSource(source=PackageSource.Archive)) == "sha"
check_output_mock.assert_has_calls([
MockCall(*sources.git(), "checkout", "--force", sources.DEFAULT_BRANCH,
cwd=local, logger=pytest.helpers.anyvar(int)),
MockCall(*sources.git(), "reset", "--quiet", "--hard", f"origin/{sources.DEFAULT_BRANCH}",
MockCall("git", "checkout", "--force", Sources.DEFAULT_BRANCH, cwd=local, logger=pytest.helpers.anyvar(int)),
MockCall("git", "reset", "--quiet", "--hard", f"origin/{Sources.DEFAULT_BRANCH}",
cwd=local, logger=pytest.helpers.anyvar(int))
])
move_mock.assert_called_once_with(local.resolve(), local)
@ -157,15 +153,15 @@ def test_fetch_relative(remote_source: RemoteSource, mocker: MockerFixture) -> N
head_mock.assert_called_once_with(local)
def test_has_remotes(sources: Sources, mocker: MockerFixture) -> None:
def test_has_remotes(mocker: MockerFixture) -> None:
"""
must ask for remotes
"""
check_output_mock = mocker.patch("ahriman.core.build_tools.sources.check_output", return_value="origin")
local = Path("local")
assert sources.has_remotes(local)
check_output_mock.assert_called_once_with(*sources.git(), "remote", cwd=local, logger=pytest.helpers.anyvar(int))
assert Sources.has_remotes(local)
check_output_mock.assert_called_once_with("git", "remote", cwd=local, logger=pytest.helpers.anyvar(int))
def test_has_remotes_empty(mocker: MockerFixture) -> None:
@ -176,7 +172,7 @@ def test_has_remotes_empty(mocker: MockerFixture) -> None:
assert not Sources.has_remotes(Path("local"))
def test_init(sources: Sources, mocker: MockerFixture) -> None:
def test_init(mocker: MockerFixture) -> None:
"""
must create empty repository at the specified path
"""
@ -187,9 +183,9 @@ def test_init(sources: Sources, mocker: MockerFixture) -> None:
commit_mock = mocker.patch("ahriman.core.build_tools.sources.Sources.commit")
local = Path("local")
sources.init(local)
check_output_mock.assert_called_once_with(*sources.git(), "init", "--quiet", "--initial-branch",
sources.DEFAULT_BRANCH, cwd=local, logger=pytest.helpers.anyvar(int))
Sources.init(local)
check_output_mock.assert_called_once_with("git", "init", "--quiet", "--initial-branch", Sources.DEFAULT_BRANCH,
cwd=local, logger=pytest.helpers.anyvar(int))
add_mock.assert_called_once_with(local, "PKGBUILD", ".SRCINFO", "local")
commit_mock.assert_called_once_with(local)
@ -271,7 +267,7 @@ def test_patch_create_with_newline(mocker: MockerFixture) -> None:
assert Sources.patch_create(Path("local"), "glob").endswith("\n")
def test_push(package_ahriman: Package, sources: Sources, mocker: MockerFixture) -> None:
def test_push(package_ahriman: Package, mocker: MockerFixture) -> None:
"""
must correctly push files to remote repository
"""
@ -281,11 +277,11 @@ def test_push(package_ahriman: Package, sources: Sources, mocker: MockerFixture)
commit_author = ("commit author", "user@host")
local = Path("local")
sources.push(local, package_ahriman.remote, "glob", commit_author=commit_author)
Sources.push(local, package_ahriman.remote, "glob", commit_author=commit_author)
add_mock.assert_called_once_with(local, "glob")
commit_mock.assert_called_once_with(local, commit_author=commit_author)
check_output_mock.assert_called_once_with(
*sources.git(), "push", "--quiet", package_ahriman.remote.git_url, package_ahriman.remote.branch,
"git", "push", "--quiet", package_ahriman.remote.git_url, package_ahriman.remote.branch,
cwd=local, logger=pytest.helpers.anyvar(int))
@ -312,7 +308,7 @@ def test_add(sources: Sources, mocker: MockerFixture) -> None:
sources.add(local, "pattern1", "pattern2")
glob_mock.assert_has_calls([MockCall("pattern1"), MockCall("pattern2")])
check_output_mock.assert_called_once_with(
*sources.git(), "add", "1", "2", "1", "2", cwd=local, logger=sources.logger
"git", "add", "1", "2", "1", "2", cwd=local, logger=sources.logger
)
@ -327,7 +323,7 @@ def test_add_intent_to_add(sources: Sources, mocker: MockerFixture) -> None:
sources.add(local, "pattern1", "pattern2", intent_to_add=True)
glob_mock.assert_has_calls([MockCall("pattern1"), MockCall("pattern2")])
check_output_mock.assert_called_once_with(
*sources.git(), "add", "--intent-to-add", "1", "2", "1", "2", cwd=local, logger=sources.logger
"git", "add", "--intent-to-add", "1", "2", "1", "2", cwd=local, logger=sources.logger
)
@ -354,8 +350,13 @@ def test_commit(sources: Sources, mocker: MockerFixture) -> None:
user, email = sources.DEFAULT_COMMIT_AUTHOR
assert sources.commit(local, message=message)
check_output_mock.assert_called_once_with(
*sources.git(), "-c", f"user.email=\"{email}\"", "-c", f"user.name=\"{user}\"",
"commit", "--quiet", "--message", message, cwd=local, logger=sources.logger
"git", "commit", "--quiet", "--message", message,
cwd=local, logger=sources.logger, environment={
"GIT_AUTHOR_NAME": user,
"GIT_AUTHOR_EMAIL": email,
"GIT_COMMITTER_NAME": user,
"GIT_COMMITTER_EMAIL": email,
}
)
@ -382,8 +383,13 @@ def test_commit_author(sources: Sources, mocker: MockerFixture) -> None:
user, email = author = ("commit author", "user@host")
assert sources.commit(Path("local"), message=message, commit_author=author)
check_output_mock.assert_called_once_with(
*sources.git(), "-c", f"user.email=\"{email}\"", "-c", f"user.name=\"{user}\"",
"commit", "--quiet", "--message", message, cwd=local, logger=sources.logger
"git", "commit", "--quiet", "--message", message,
cwd=local, logger=sources.logger, environment={
"GIT_AUTHOR_NAME": user,
"GIT_AUTHOR_EMAIL": email,
"GIT_COMMITTER_NAME": user,
"GIT_COMMITTER_EMAIL": email,
}
)
@ -398,8 +404,13 @@ def test_commit_autogenerated_message(sources: Sources, mocker: MockerFixture) -
assert sources.commit(Path("local"))
user, email = sources.DEFAULT_COMMIT_AUTHOR
check_output_mock.assert_called_once_with(
*sources.git(), "-c", f"user.email=\"{email}\"", "-c", f"user.name=\"{user}\"",
"commit", "--quiet", "--message", pytest.helpers.anyvar(str, strict=True), cwd=local, logger=sources.logger
"git", "commit", "--quiet", "--message", pytest.helpers.anyvar(str, strict=True),
cwd=local, logger=sources.logger, environment={
"GIT_AUTHOR_NAME": user,
"GIT_AUTHOR_EMAIL": email,
"GIT_COMMITTER_NAME": user,
"GIT_COMMITTER_EMAIL": email,
}
)
@ -411,7 +422,7 @@ def test_diff(sources: Sources, mocker: MockerFixture) -> None:
local = Path("local")
assert sources.diff(local)
check_output_mock.assert_called_once_with(*sources.git(), "diff", cwd=local, logger=sources.logger)
check_output_mock.assert_called_once_with("git", "diff", cwd=local, logger=sources.logger)
def test_diff_specific(sources: Sources, mocker: MockerFixture) -> None:
@ -422,7 +433,7 @@ def test_diff_specific(sources: Sources, mocker: MockerFixture) -> None:
local = Path("local")
assert sources.diff(local, "hash")
check_output_mock.assert_called_once_with(*sources.git(), "diff", "hash", cwd=local, logger=sources.logger)
check_output_mock.assert_called_once_with("git", "diff", "hash", cwd=local, logger=sources.logger)
def test_fetch_until(sources: Sources, mocker: MockerFixture) -> None:
@ -439,12 +450,10 @@ def test_fetch_until(sources: Sources, mocker: MockerFixture) -> None:
local = Path("local")
sources.fetch_until(local, branch="master", commit_sha="sha")
check_output_mock.assert_has_calls([
MockCall(*sources.git(), "fetch", "--quiet", "--depth", "1", "origin", "master",
cwd=local, logger=sources.logger),
MockCall(*sources.git(), "cat-file", "-e", "sha", cwd=local, logger=sources.logger),
MockCall(*sources.git(), "fetch", "--quiet", "--depth", "2", "origin", "master",
cwd=local, logger=sources.logger),
MockCall(*sources.git(), "cat-file", "-e", "sha", cwd=local, logger=sources.logger),
MockCall("git", "fetch", "--quiet", "--depth", "1", "origin", "master", cwd=local, logger=sources.logger),
MockCall("git", "cat-file", "-e", "sha", cwd=local, logger=sources.logger),
MockCall("git", "fetch", "--quiet", "--depth", "2", "origin", "master", cwd=local, logger=sources.logger),
MockCall("git", "cat-file", "-e", "sha", cwd=local, logger=sources.logger),
])
@ -457,9 +466,8 @@ def test_fetch_until_first(sources: Sources, mocker: MockerFixture) -> None:
local = Path("local")
sources.fetch_until(local, branch="master")
check_output_mock.assert_has_calls([
MockCall(*sources.git(), "fetch", "--quiet", "--depth", "1", "origin", "master",
cwd=local, logger=sources.logger),
MockCall(*sources.git(), "cat-file", "-e", "HEAD", cwd=local, logger=sources.logger),
MockCall("git", "fetch", "--quiet", "--depth", "1", "origin", "master", cwd=local, logger=sources.logger),
MockCall("git", "cat-file", "-e", "HEAD", cwd=local, logger=sources.logger),
])
@ -472,27 +480,11 @@ def test_fetch_until_all_branches(sources: Sources, mocker: MockerFixture) -> No
local = Path("local")
sources.fetch_until(local)
check_output_mock.assert_has_calls([
MockCall(*sources.git(), "fetch", "--quiet", "--depth", "1", cwd=local, logger=sources.logger),
MockCall(*sources.git(), "cat-file", "-e", "HEAD", cwd=local, logger=sources.logger),
MockCall("git", "fetch", "--quiet", "--depth", "1", cwd=local, logger=sources.logger),
MockCall("git", "cat-file", "-e", "HEAD", cwd=local, logger=sources.logger),
])
def test_git(sources: Sources) -> None:
"""
must correctly generate git command
"""
assert sources.git() == ["git", "-c", "init.defaultBranch=\"master\""]
def test_git_overrides(sources: Sources) -> None:
"""
must correctly generate git command with additional settings
"""
assert sources.git({"user.email": "ahriman@localhost"}) == [
"git", "-c", "init.defaultBranch=\"master\"", "-c", "user.email=\"ahriman@localhost\""
]
def test_has_changes(sources: Sources, mocker: MockerFixture) -> None:
"""
must correctly identify if there are changes
@ -501,12 +493,12 @@ def test_has_changes(sources: Sources, mocker: MockerFixture) -> None:
check_output_mock = mocker.patch("ahriman.core.build_tools.sources.check_output", return_value="M a.txt")
assert sources.has_changes(local)
check_output_mock.assert_called_once_with(*sources.git(), "diff", "--cached", "--name-only",
check_output_mock.assert_called_once_with("git", "diff", "--cached", "--name-only",
cwd=local, logger=sources.logger)
check_output_mock = mocker.patch("ahriman.core.build_tools.sources.check_output", return_value="")
assert not sources.has_changes(local)
check_output_mock.assert_called_once_with(*sources.git(), "diff", "--cached", "--name-only",
check_output_mock.assert_called_once_with("git", "diff", "--cached", "--name-only",
cwd=local, logger=sources.logger)
@ -518,7 +510,7 @@ def test_head(sources: Sources, mocker: MockerFixture) -> None:
local = Path("local")
assert sources.head(local) == "sha"
check_output_mock.assert_called_once_with(*sources.git(), "rev-parse", "HEAD", cwd=local, logger=sources.logger)
check_output_mock.assert_called_once_with("git", "rev-parse", "HEAD", cwd=local, logger=sources.logger)
def test_head_specific(sources: Sources, mocker: MockerFixture) -> None:
@ -529,7 +521,7 @@ def test_head_specific(sources: Sources, mocker: MockerFixture) -> None:
local = Path("local")
assert sources.head(local, "master") == "sha"
check_output_mock.assert_called_once_with(*sources.git(), "rev-parse", "master", cwd=local, logger=sources.logger)
check_output_mock.assert_called_once_with("git", "rev-parse", "master", cwd=local, logger=sources.logger)
def test_move(sources: Sources, mocker: MockerFixture) -> None:
@ -562,7 +554,7 @@ def test_patch_apply(sources: Sources, mocker: MockerFixture) -> None:
local = Path("local")
sources.patch_apply(local, patch)
check_output_mock.assert_called_once_with(
*sources.git(), "apply", "--ignore-space-change", "--ignore-whitespace",
"git", "apply", "--ignore-space-change", "--ignore-whitespace",
cwd=local, input_data=patch.value, logger=sources.logger
)

View File

@ -4,7 +4,6 @@ from pytest_mock import MockerFixture
from ahriman.core.configuration import Configuration
from ahriman.core.database import SQLite
from ahriman.models.repository_id import RepositoryId
def test_load(configuration: Configuration, mocker: MockerFixture) -> None:
@ -36,7 +35,7 @@ def test_init_skip_migration(database: SQLite, configuration: Configuration, moc
migrate_schema_mock.assert_not_called()
def test_package_clear(database: SQLite, repository_id: RepositoryId, mocker: MockerFixture) -> None:
def test_package_clear(database: SQLite, mocker: MockerFixture) -> None:
"""
must clear package data
"""
@ -45,14 +44,12 @@ def test_package_clear(database: SQLite, repository_id: RepositoryId, mocker: Mo
logs_mock = mocker.patch("ahriman.core.database.SQLite.logs_remove")
changes_mock = mocker.patch("ahriman.core.database.SQLite.changes_remove")
dependencies_mock = mocker.patch("ahriman.core.database.SQLite.dependencies_remove")
package_mock = mocker.patch("ahriman.core.database.SQLite.package_remove")
tree_clear_mock = mocker.patch("ahriman.models.repository_paths.RepositoryPaths.tree_clear")
database.package_clear("package", repository_id)
build_queue_mock.assert_called_once_with("package", repository_id)
patches_mock.assert_called_once_with("package", None)
logs_mock.assert_called_once_with("package", None, repository_id)
changes_mock.assert_called_once_with("package", repository_id)
dependencies_mock.assert_called_once_with("package", repository_id)
package_mock.assert_called_once_with("package", repository_id)
database.package_clear("package")
build_queue_mock.assert_called_once_with("package")
patches_mock.assert_called_once_with("package", [])
logs_mock.assert_called_once_with("package", None)
changes_mock.assert_called_once_with("package")
dependencies_mock.assert_called_once_with("package")
tree_clear_mock.assert_called_once_with("package")

View File

@ -158,7 +158,7 @@ def test_package_remove(local_client: LocalClient, package_ahriman: Package, moc
"""
package_mock = mocker.patch("ahriman.core.database.SQLite.package_clear")
local_client.package_remove(package_ahriman.base)
package_mock.assert_called_once_with(package_ahriman.base, local_client.repository_id)
package_mock.assert_called_once_with(package_ahriman.base)
def test_package_status_update(local_client: LocalClient, package_ahriman: Package, mocker: MockerFixture) -> None:

View File

@ -101,11 +101,13 @@ def test_package_remove(watcher: Watcher, package_ahriman: Package, mocker: Mock
must remove package base
"""
cache_mock = mocker.patch("ahriman.core.status.local_client.LocalClient.package_remove")
logs_mock = mocker.patch("ahriman.core.status.watcher.Watcher.package_logs_remove", create=True)
watcher._known = {package_ahriman.base: (package_ahriman, BuildStatus())}
watcher.package_remove(package_ahriman.base)
assert not watcher._known
cache_mock.assert_called_once_with(package_ahriman.base)
logs_mock.assert_called_once_with(package_ahriman.base, None)
def test_package_remove_unknown(watcher: Watcher, package_ahriman: Package, mocker: MockerFixture) -> None:

View File

@ -7,17 +7,10 @@ def test_id() -> None:
"""
must correctly generate id
"""
assert RepositoryId("", "").id == ""
assert RepositoryId("arch", "repo").id == "arch-repo"
def test_id_empty() -> None:
"""
must raise exception on empty identifier
"""
with pytest.raises(ValueError):
assert RepositoryId("", "").id
def test_is_empty() -> None:
"""
must check if repository id is empty or not

View File

@ -201,7 +201,7 @@ def test_service_not_found(base: BaseView) -> None:
must raise HTTPNotFound if no repository found
"""
with pytest.raises(HTTPNotFound):
base.service(RepositoryId("repo", "arch"))
base.service(RepositoryId("", ""))
def test_service_package(base: BaseView, repository_id: RepositoryId, mocker: MockerFixture) -> None:

View File

@ -11,7 +11,6 @@ flags = --implicit-reexport --strict --allow-untyped-decorators --allow-subclass
[pytest]
addopts = --cov=ahriman --cov-report=term-missing:skip-covered --no-cov-on-fail --cov-fail-under=100 --spec
asyncio_default_fixture_loop_scope = function
asyncio_mode = auto
spec_test_format = {result} {docstring_summary}