Compare commits

..

3 Commits

Author SHA1 Message Date
aaededa303 docs: improve waiter classes docs 2024-08-22 18:28:26 +03:00
3f5a3b8a1a feat: add ability to suppress git hints
It can be done by setting options in command. The commit author/email is
also now using this logic
2024-08-22 18:17:56 +03:00
3a78b42df6 feat: add blacklisted paths to implicit dependencies processing
It has been found that in some cases additional packages have been added
as dependencies, like usr/share/applications, usr/lib/cmake, etc

This commit adds an ability to blacklist specific paths from processing
2024-08-21 15:26:43 +03:00
26 changed files with 6770 additions and 7270 deletions

View File

@ -132,7 +132,7 @@ Again, the most checks can be performed by `tox` command, though some additional
* For any path interactions `pathlib.Path` must be used.
* Configuration interactions must go through `ahriman.core.configuration.Configuration` class instance.
* In case if class load requires some actions, it is recommended to create class method which can be used for class instantiating.
* The most (expected) exceptions must be handled and printed to log, allowing service to continue work. However, fatal and (in some cases) unexpected exceptions may lead to the application termination.
* The code must follow the exception safety, unless it is explicitly asked by end user. It means that most exceptions must be handled and printed to log, no other actions must be done (e.g. raising another exception).
* Exceptions without parameters should be raised without parentheses, e.g.:
```python

View File

@ -40,5 +40,3 @@ The application provides reasonable defaults which allow to use it out-of-box; h
* [Build status page](https://ahriman-demo.arcanis.me). You can log in as `demo` user by using `demo` password. However, you will not be able to run tasks. [HTTP API documentation](https://ahriman-demo.arcanis.me/api-docs) is also available.
* [Repository index](https://repo.arcanis.me/arcanisrepo/x86_64/).
* [Telegram feed](https://t.me/arcanisrepo).
Do you have any success story? You can [share it](https://github.com/arcan1s/ahriman/issues/new?template=04-discussion.md)!

File diff suppressed because it is too large Load Diff

Before

Width:  |  Height:  |  Size: 1.2 MiB

After

Width:  |  Height:  |  Size: 1.1 MiB

View File

@ -228,14 +228,6 @@ ahriman.models.result module
:no-undoc-members:
:show-inheritance:
ahriman.models.scan\_paths module
---------------------------------
.. automodule:: ahriman.models.scan_paths
:members:
:no-undoc-members:
:show-inheritance:
ahriman.models.sign\_settings module
------------------------------------

View File

@ -370,16 +370,7 @@ TL;DR
You can even rebuild the whole repository (which is particular useful in case if you would like to change packager) if you do not supply ``--depends-on`` option. This action will automatically increment ``pkgrel`` value; in case if you don't want to, the ``--no-increment`` option has to be supplied.
However, note that you do not need to rebuild repository in case if you just changed signing option, just use ``repo-sign`` command instead.
Automated broken dependencies detection
"""""""""""""""""""""""""""""""""""""""
After the success build the application extracts all linked libraries and used directories and stores them in database. During the check process, the application extracts pacman databases and checks if file names have been changed (e.g. new python release caused ``/usr/lib/python3.x`` directory renaming to ``/usr/lib/python3.y`` or soname for a linked library has been changed). In case if broken dependencies have been detected, the package will be added to the rebuild queue.
In order to disable this check completely, the ``--no-check-files`` flag can be used.
In addition, there is possibility to control paths which will be used for checking, by using options ``build.allowed_scan_paths`` and ``build.blacklisted_scan_paths``. Leaving ``build.allowed_scan_paths`` blank will effectively disable any check too.
However, note that you do not need to rebuild repository in case if you just changed signing option, just use ``repo-sign`` command instead.
How to install built packages
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

View File

@ -1,7 +1,7 @@
# Maintainer: Evgeniy Alekseev
pkgname='ahriman'
pkgver=2.14.1
pkgver=2.13.8
pkgrel=1
pkgdesc="ArcH linux ReposItory MANager"
arch=('any')

View File

@ -1,4 +1,4 @@
.TH AHRIMAN "1" "2024\-09\-04" "ahriman" "Generated Python Manual"
.TH AHRIMAN "1" "2024\-05\-12" "ahriman" "Generated Python Manual"
.SH NAME
ahriman
.SH SYNOPSIS
@ -391,7 +391,7 @@ PKGBUILD variable or function name. If variable is a function, it must end with
path to file which contains function or variable value. If not set, the value will be read from stdin
.SH COMMAND \fI\,'ahriman patch\-list'\/\fR
usage: ahriman patch\-list [\-h] [\-e] [\-v VARIABLE] package
usage: ahriman patch\-list [\-h] [\-e] [\-v VARIABLE] [package]
list available patches for the package

View File

@ -86,7 +86,7 @@ _shtab_ahriman_options=(
{-a,--architecture}"[filter by target architecture (default\: None)]:architecture:"
{-c,--configuration}"[configuration path (default\: \/etc\/ahriman.ini)]:configuration:"
"--force[force run, remove file lock (default\: False)]"
{-l,--lock}"[lock file (default\: ahriman.pid)]:lock:"
{-l,--lock}"[lock file (default\: \/tmp\/ahriman.lock)]:lock:"
"--log-handler[explicit log handler specification. If none set, the handler will be guessed from environment (default\: None)]:log_handler:(console syslog journald)"
{-q,--quiet}"[force disable any logging (default\: False)]"
{--report,--no-report}"[force enable or disable reporting to web service (default\: True)]:report:"
@ -280,7 +280,7 @@ _shtab_ahriman_patch_list_options=(
"(- : *)"{-h,--help}"[show this help message and exit]"
{-e,--exit-code}"[return non-zero exit status if result is empty (default\: False)]"
"*"{-v,--variable}"[if set, show only patches for specified PKGBUILD variables (default\: None)]:variable:"
":package base:"
":package base (default\: None):"
)
_shtab_ahriman_patch_remove_options=(

View File

@ -17,4 +17,4 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
__version__ = "2.14.1"
__version__ = "2.13.8"

View File

@ -75,9 +75,7 @@ class Lock(LazyLogging):
"""
self.path: Path | None = None
if args.lock is not None:
self.path = args.lock
if not repository_id.is_empty:
self.path = self.path.with_stem(f"{args.lock.stem}_{repository_id.id}")
self.path = args.lock.with_stem(f"{args.lock.stem}_{repository_id.id}")
if not self.path.is_absolute():
# prepend full path to the lock file
self.path = Path("/") / "run" / "ahriman" / self.path

View File

@ -43,7 +43,7 @@ class Pacman(LazyLogging):
configuration(Configuration): configuration instance
refresh_database(PacmanSynchronization): synchronize local cache to remote
repository_id(RepositoryId): repository unique identifier
repository_paths(RepositoryPaths): repository paths instance
repository_path(RepositoryPaths): repository paths instance
"""
def __init__(self, repository_id: RepositoryId, configuration: Configuration, *,
@ -188,8 +188,8 @@ class Pacman(LazyLogging):
Returns:
dict[str, set[str]]: map of package name to its list of files
"""
def extract(tar: tarfile.TarFile, versions: dict[str, str]) -> Generator[tuple[str, set[str]], None, None]:
for package_name, version in versions.items():
def extract(tar: tarfile.TarFile, package_names: dict[str, str]) -> Generator[tuple[str, set[str]], None, None]:
for package_name, version in package_names.items():
path = Path(f"{package_name}-{version}") / "files"
try:
content = tar.extractfile(str(path))

View File

@ -59,8 +59,7 @@ class PacmanDatabase(SyncHttpClient):
self.sync_files_database = configuration.getboolean("alpm", "sync_files_database")
@staticmethod
def copy(remote_path: Path, local_path: Path) -> None:
def copy(self, remote_path: Path, local_path: Path) -> None:
"""
copy local database file

View File

@ -27,7 +27,6 @@ from ahriman.core.configuration import Configuration
from ahriman.core.database.migrations import Migrations
from ahriman.core.database.operations import AuthOperations, BuildOperations, ChangesOperations, \
DependenciesOperations, LogsOperations, PackageOperations, PatchOperations
from ahriman.models.repository_id import RepositoryId
# pylint: disable=too-many-ancestors
@ -103,26 +102,23 @@ class SQLite(
self.with_connection(lambda connection: Migrations.migrate(connection, configuration))
paths.chown(self.path)
def package_clear(self, package_base: str, repository_id: RepositoryId | None = None) -> None:
def package_clear(self, package_base: str) -> None:
"""
completely remove package from all tables
Args:
package_base(str): package base to remove
repository_id(RepositoryId, optional): repository unique identifier override (Default value = None)
Examples:
This method completely removes the package from all tables and must be used, e.g. on package removal::
>>> database.package_clear("ahriman")
"""
self.build_queue_clear(package_base, repository_id)
self.patches_remove(package_base, None)
self.logs_remove(package_base, None, repository_id)
self.changes_remove(package_base, repository_id)
self.dependencies_remove(package_base, repository_id)
self.package_remove(package_base, repository_id)
self.build_queue_clear(package_base)
self.patches_remove(package_base, [])
self.logs_remove(package_base, None)
self.changes_remove(package_base)
self.dependencies_remove(package_base)
# remove local cache too
self._repository_paths.tree_clear(package_base)

View File

@ -310,7 +310,7 @@ class Client:
def set_unknown(self, package: Package) -> None:
"""
set package status to unknown. Unlike other methods, this method also checks if package is known,
and - in case if it is - it silently skips update
and - in case if it is - it silently skips updatd
Args:
package(Package): current package properties

View File

@ -184,7 +184,7 @@ class LocalClient(Client):
Args:
package_base(str): package base to remove
"""
self.database.package_clear(package_base, self.repository_id)
self.database.package_clear(package_base)
def package_status_update(self, package_base: str, status: BuildStatusEnum) -> None:
"""

View File

@ -140,6 +140,7 @@ class Watcher(LazyLogging):
with self._lock:
self._known.pop(package_base, None)
self.client.package_remove(package_base)
self.package_logs_remove(package_base, None)
def package_status_update(self, package_base: str, status: BuildStatusEnum) -> None:
"""

View File

@ -41,12 +41,9 @@ class RepositoryId:
Returns:
str: unique id for this repository
Raises:
ValueError: if repository identifier is empty
"""
if self.is_empty:
raise ValueError("Repository ID is called on empty repository identifier")
return ""
return f"{self.architecture}-{self.name}" # basically the same as used for command line
@property

View File

@ -113,7 +113,7 @@ class RepositoryPaths(LazyLogging):
Returns:
Path: full patch to devtools chroot directory
"""
# for the chroot directory devtools will create own tree, and we don't have to specify architecture here
# for the chroot directory devtools will create own tree, and we don"t have to specify architecture here
return self.root / "chroot" / self.repository_id.name
@property

View File

@ -14,7 +14,6 @@ from ahriman.core.configuration import Configuration
from ahriman.core.exceptions import DuplicateRunError, UnsafeRunError
from ahriman.models.build_status import BuildStatus, BuildStatusEnum
from ahriman.models.internal_status import InternalStatus
from ahriman.models.repository_id import RepositoryId
def test_path(args: argparse.Namespace, configuration: Configuration) -> None:
@ -31,8 +30,6 @@ def test_path(args: argparse.Namespace, configuration: Configuration) -> None:
args.lock = Path("ahriman.pid")
assert Lock(args, repository_id, configuration).path == Path("/run/ahriman/ahriman_x86_64-aur-clone.pid")
assert Lock(args, RepositoryId("", ""), configuration).path == Path("/run/ahriman/ahriman.pid")
with pytest.raises(ValueError):
args.lock = Path("/")
assert Lock(args, repository_id, configuration).path # special case

View File

@ -8,12 +8,12 @@ from ahriman.core.alpm.pacman_database import PacmanDatabase
from ahriman.core.exceptions import PacmanError
def test_copy(mocker: MockerFixture) -> None:
def test_copy(pacman_database: PacmanDatabase, mocker: MockerFixture) -> None:
"""
must copy loca database file
"""
copy_mock = mocker.patch("shutil.copy")
PacmanDatabase.copy(Path("remote"), Path("local"))
pacman_database.copy(Path("remote"), Path("local"))
copy_mock.assert_called_once_with(Path("remote"), Path("local"))

View File

@ -4,7 +4,6 @@ from pytest_mock import MockerFixture
from ahriman.core.configuration import Configuration
from ahriman.core.database import SQLite
from ahriman.models.repository_id import RepositoryId
def test_load(configuration: Configuration, mocker: MockerFixture) -> None:
@ -36,7 +35,7 @@ def test_init_skip_migration(database: SQLite, configuration: Configuration, moc
migrate_schema_mock.assert_not_called()
def test_package_clear(database: SQLite, repository_id: RepositoryId, mocker: MockerFixture) -> None:
def test_package_clear(database: SQLite, mocker: MockerFixture) -> None:
"""
must clear package data
"""
@ -45,14 +44,12 @@ def test_package_clear(database: SQLite, repository_id: RepositoryId, mocker: Mo
logs_mock = mocker.patch("ahriman.core.database.SQLite.logs_remove")
changes_mock = mocker.patch("ahriman.core.database.SQLite.changes_remove")
dependencies_mock = mocker.patch("ahriman.core.database.SQLite.dependencies_remove")
package_mock = mocker.patch("ahriman.core.database.SQLite.package_remove")
tree_clear_mock = mocker.patch("ahriman.models.repository_paths.RepositoryPaths.tree_clear")
database.package_clear("package", repository_id)
build_queue_mock.assert_called_once_with("package", repository_id)
patches_mock.assert_called_once_with("package", None)
logs_mock.assert_called_once_with("package", None, repository_id)
changes_mock.assert_called_once_with("package", repository_id)
dependencies_mock.assert_called_once_with("package", repository_id)
package_mock.assert_called_once_with("package", repository_id)
database.package_clear("package")
build_queue_mock.assert_called_once_with("package")
patches_mock.assert_called_once_with("package", [])
logs_mock.assert_called_once_with("package", None)
changes_mock.assert_called_once_with("package")
dependencies_mock.assert_called_once_with("package")
tree_clear_mock.assert_called_once_with("package")

View File

@ -158,7 +158,7 @@ def test_package_remove(local_client: LocalClient, package_ahriman: Package, moc
"""
package_mock = mocker.patch("ahriman.core.database.SQLite.package_clear")
local_client.package_remove(package_ahriman.base)
package_mock.assert_called_once_with(package_ahriman.base, local_client.repository_id)
package_mock.assert_called_once_with(package_ahriman.base)
def test_package_status_update(local_client: LocalClient, package_ahriman: Package, mocker: MockerFixture) -> None:

View File

@ -101,11 +101,13 @@ def test_package_remove(watcher: Watcher, package_ahriman: Package, mocker: Mock
must remove package base
"""
cache_mock = mocker.patch("ahriman.core.status.local_client.LocalClient.package_remove")
logs_mock = mocker.patch("ahriman.core.status.watcher.Watcher.package_logs_remove", create=True)
watcher._known = {package_ahriman.base: (package_ahriman, BuildStatus())}
watcher.package_remove(package_ahriman.base)
assert not watcher._known
cache_mock.assert_called_once_with(package_ahriman.base)
logs_mock.assert_called_once_with(package_ahriman.base, None)
def test_package_remove_unknown(watcher: Watcher, package_ahriman: Package, mocker: MockerFixture) -> None:

View File

@ -7,17 +7,10 @@ def test_id() -> None:
"""
must correctly generate id
"""
assert RepositoryId("", "").id == ""
assert RepositoryId("arch", "repo").id == "arch-repo"
def test_id_empty() -> None:
"""
must raise exception on empty identifier
"""
with pytest.raises(ValueError):
assert RepositoryId("", "").id
def test_is_empty() -> None:
"""
must check if repository id is empty or not

View File

@ -201,7 +201,7 @@ def test_service_not_found(base: BaseView) -> None:
must raise HTTPNotFound if no repository found
"""
with pytest.raises(HTTPNotFound):
base.service(RepositoryId("repo", "arch"))
base.service(RepositoryId("", ""))
def test_service_package(base: BaseView, repository_id: RepositoryId, mocker: MockerFixture) -> None:

View File

@ -11,7 +11,6 @@ flags = --implicit-reexport --strict --allow-untyped-decorators --allow-subclass
[pytest]
addopts = --cov=ahriman --cov-report=term-missing:skip-covered --no-cov-on-fail --cov-fail-under=100 --spec
asyncio_default_fixture_loop_scope = function
asyncio_mode = auto
spec_test_format = {result} {docstring_summary}