Commit Graph

178 Commits

Author SHA1 Message Date
70e9e3e822 migrate to the latest devtools & archlinux git changes
This migration includes
* removal of community repository as it is no more
* fixed paths for devtools configurations
* migration of archlinux packaging git url
2023-05-22 16:51:27 +03:00
2ff56965d9
PEP-673 use Self as return type for classmethods (#94)
* PEP-673 use Self as return type for classmethods

* add dummy test file

* remove python3.10 compat
2023-05-04 03:28:08 +03:00
abf29083c4
PEP-585 complaint: remove type aliases (#93) 2023-04-12 04:09:28 +03:00
58379e7bf3 optimize imports 2023-04-06 00:24:39 +03:00
df23be9269 gracefully terminate web server
In previous revisions server was terminated by itself, thus no lock or
socket was removed. In new version, graceful termination of the queue
has been added as well as server now handles singals
2023-03-06 01:13:41 +02:00
13faf66bdb add more validation rules 2023-02-23 15:18:56 +02:00
37e57c13c8
update dependencies before build (#91)
Old implementation has used add step in order to fetch dependencies,
which could lead to build errors in case if dependency list was updated.

New solution uses dependencies which are declared at current version and
fetch them (if required and if enabled) before update process.

Closes #90
2023-02-12 06:02:30 +03:00
3a4e8f4d97 mask mypy warning
The newest mypy produces the following warning:

src/ahriman/application/handlers/search.py:43: error: Non-overlapping identity check (left operand type: "Union[_DefaultFactory[Any], Literal[_MISSING_TYPE.MISSING]]", right operand type: "Type[List[Any]]")  [comparison-overlap]

which is more likely caused by updated dataclass models to protoocol (however decorators are still calllable). This commit masks problematic line from checking
2023-02-09 22:46:08 +02:00
4db8ad8e8d hide passwords and secrets from repo-config subcommand by default 2023-02-05 16:44:48 +02:00
917ec48be5 handle architecture specific fields for dependencies
This change requires srcinfo at least 0.1.2 version. Unfortunatelly aur
api don't support architecture specific arrays for now, so we just leave
it as is

Closes #82
2023-01-31 14:34:09 +02:00
5af84955ac
calculate dependencies based on package information (#89) 2023-01-30 17:28:05 +03:00
9e4e3b701b enable lock for web service 2023-01-18 01:39:55 +02:00
616a1950ae add ability to override pacman mirror in devtools configuration
This commit also extends configuration of the multilib option, adding
the ability to exlcude multilib repository from repositories list

Note, that in order to support repository list and mirror correctly,
alpm configuration section is now architectture specific
2023-01-15 18:05:15 +02:00
d7356926c4 consider vcs flag and vcs_allowed_age during local packages update 2023-01-12 15:37:05 +02:00
43a7d09cab add ability to check for service updates 2023-01-12 15:37:05 +02:00
77954b988b subcommand review
Some commands have been moved to another group and thus having another
default name (old subcommands are still available...for now):

* daemon -> repo-daemon
* key-import -> service-key-import
* repo-clean -> service-clean
* repo-config -> service-config
* repo-config-validate -> service-config-validate
* repo-setup -> service-setup
* repo-shell -> service-shell
* version -> help-version
2023-01-12 15:37:05 +02:00
0239fb50b6 expose trigger configuration schema
Note that this commit contains the following breaking changes:

* remote pull and remote push triggers are now enabled by default (with
  empty target list)
* remote pull and remote push triggers now require target option to be
  set (old behaviour had fallback on `gitremote`)
* validation is now considered to be stable, so it is enabled by default
  in docker image (can be disabled however)
2023-01-10 03:33:52 +02:00
d942a70272
add config validator subcommand (#80)
* add config validator subcommand

* add --exit-code flag

* docs & faq update
2023-01-09 18:22:29 +03:00
014007ade3 review unsafe commands access
Some commands were made unsafe in old versions, but nowadays they can be
run without having special privileges.

There was also a bug in which status commands were not available if you
are not ahriman user and unix socket is used. It has been fixed by
switching to manual socket creation (see also
https://github.com/aio-libs/aiohttp/issues/4155)
2023-01-04 17:45:44 +02:00
04e5a263b7 add notes about documentation and methods inside class
Because I always forget which way I used before
2023-01-03 01:53:10 +02:00
7de94a8daa add repo-structure subcommand
This commit also changes Tree class, replacing load method by resolve
2022-12-27 10:35:03 +02:00
8864855c14
write patches via gitremote push trigger (#79)
* write patches via gitremote push trigger

* implement context variables intead of custom database class
2022-12-25 02:10:38 +03:00
7aa91f9e2e do not trigger update on sign command 2022-12-02 01:41:23 +02:00
01eda513cf improve setup command by --makeflags-jobs argument and fix repository sign on creation 2022-12-02 01:41:23 +02:00
0161617e36 implement support of unix socket for server
This feature can be used for unauthorized access to apis - e.g. for
reporting service if it is run on the same machine. Since now it becomes
recommended way for the interprocess communication, thus some options
(e.g. creating user with as-service flag) are no longer available now
2022-12-02 01:41:23 +02:00
4811dec759 do not invoke configuration write in case if no salt or user was written 2022-12-02 01:41:23 +02:00
09623c20d5 add rebuild implementation to interface 2022-12-02 01:41:23 +02:00
5073c80af1 add key-import button to interface 2022-12-02 01:41:23 +02:00
78e6b48c24 ask users to repeat password
In case if password is asked via getpass, it is possible to make typo
and user will not see the mistake. In order to avoid it, additional
confirmation has been added
2022-11-22 02:19:37 +02:00
137d62e2f8
Extended package status page (#76)
* implement log storage at backend
* handle process id during removal. During one process we can write logs from different packages in different times (e.g. check and update later) and we would like to store all logs belong to the same process
* set package context in main functions
* implement logs support in interface
* filter out logs posting http logs
* add timestamp to log records
* hide getting logs under reporter permission

List of breaking changes:

* `ahriman.core.lazy_logging.LazyLogging` has been renamed to `ahriman.core.log.LazyLogging`
* `ahriman.core.configuration.Configuration.from_path` does not have `quiet` attribute now
* `ahriman.core.configuration.Configuration` class does not have `load_logging` method now
* `ahriman.core.status.client.Client.load` requires `report` argument now
2022-11-22 02:58:22 +03:00
0cd07afa0f use intersection of from_database and depends_on filters for the rebuild subcommand
Old logic used OR condition, i.e. if set from-database, it would ignore
the --depends-on flag. In new logic it calculates dependencies based on
the package list, which can be retrieved from database
2022-11-11 16:02:19 +02:00
9556779ee2 add exec to shell subcommand 2022-11-10 18:52:55 +02:00
0eadef597a
replace several store_true keys to booleanoptionalaction alternative (#74) 2022-11-10 18:34:01 +03:00
e58ccdc8ad write .makepkg.conf to home dir instead of repository root (#72) 2022-11-08 17:02:13 +02:00
2a07356d24
add ability to use ahriman pacman database instead of system one (#71)
By default this feature is enabled. On the first run it will copy (if
exists) databases from filesystem to local cache (one per each
architecture). Later it will use this cache for all alpm operations. In
order to update this cache, some commands (mainly package building)
provide `-y`/`--refresh` option which has same semantics as pacman -Sy
does.

Note however that due to extending `Pacman` class some methods were
renamed in order to be more descriptive:
* `Pacman.all_packages` -> `Pacman.packages`
* `Pacman.get` -> `Pacman.package_get`

This commit also adds multilib repository to the default docker image
which was missed.
2022-11-08 17:26:51 +03:00
43c553a3db improve repo-setup command
* Move devtools executable to ahriman home, because we don't really
  need to use executable inside root
* Use named sudoers file instead of single file. It will allow easily to
  remove file as well as use setup command for several
  repositories/architectures
2022-11-06 14:25:44 +02:00
4f35ec6542 fix update handler tests
If daemon tests are run before the update handler it causes test
failure, because there are other calls
2022-10-31 11:51:12 +02:00
b97c8928e1 add daemon subcommand
This command emulates default systemd timer and can be useful in docker
container in order to run 24/7
2022-10-31 01:38:01 +02:00
649df81aa5
implement single-function patches (#69) 2022-10-30 03:11:03 +03:00
342b3cb652
Add gitremote triggers (#68)
* add gitremote pull trigger

* add push gitremote trigger

* docs update
2022-10-18 01:46:27 +03:00
fc0d8387df extend triggers to on_start and on_stop methods
This commit also replaces old run method to new on_result
2022-09-26 01:22:54 +03:00
1912790ccc Make optional dependencies trully optional (#67)
The issue appears when there is no boto, jinja and some other libraries
are not installed because the classes which use these libraries are
still being imported inside the package file. The fix removes those
imports from package root, because they should not be here, in fact,
content of report and upload packages must be imported only inside the
trigger class and only if they are actually required

This commit also adds setuptools as required dependency since it is used
for some parsers (previously it was provided dependency)
2022-09-11 01:44:06 +03:00
6633766cc3 frozen dataclasses 2022-07-26 14:40:28 +03:00
cbbe2b01e4 fix case with package name which cannot be downloaded
(without special settings)

The issue appears if file or its version contains one of special URI
characters, e.g. +. Theu will be interpreted as query parameters by
(some) servers (e.g. S3 works in this way). In this commit we rename
archive to the one with safe name.
2022-06-27 18:53:48 +03:00
f4131b8cd7 add shell and version parser 2022-05-27 18:34:07 +03:00
47de715d7d
dynamic html load (#63)
* dynamic html load
* split by classes
2022-05-20 22:29:36 +03:00
5674b7b388 add ability to run only speicifed triggers from command line
This commit also restores repo-report and repo-sync subcommands
2022-05-12 18:43:36 +03:00
99874845b5 triggers implementation (#62) 2022-05-09 20:00:20 +03:00
b6db2a8035 fix error with missing sources
In case if package has local cache it will fail to load because no
remote source set. Particially this case can be observed during tree
load
2022-05-08 03:56:54 +03:00
98910240dd shorten public imports 2022-05-06 04:08:05 +03:00
b7debddaea
Complete official repository support (#59) 2022-05-03 00:49:32 +03:00
1cfc751d21 rename classes in order to make documentation reference work 2022-04-18 05:30:02 +03:00
d90f417cae
Docstring update (#58)
* migrate docstrings from reST to google format

* add raises note

Also change behaviour of the `from_option` method to fallback to
disabled instead of raising exception on unknown option

* fix part of warnings for sphinx

* make identation a bit more readable

* review fixes

* add verbose description for properties to make them parsed by sphinx extenstion

* add demo sphinx generator
2022-04-17 20:25:28 +03:00
cb63bc08ff add backup and restore subcommands 2022-04-10 21:34:34 +03:00
6551c8d983 merge restore to rebuild commannd 2022-04-10 01:51:12 +03:00
2536b8dc1f add support of repository restoration 2022-04-07 04:49:07 +03:00
e200ac9776 add support of officiall repositories api 2022-04-07 04:19:37 +03:00
6946745153 fix descriptions 2022-04-06 01:48:03 +03:00
e909e4f570
add configurable exit codes to some commands (#55) 2022-04-01 18:30:11 +03:00
06ec16ac77 create user in docker container
also fix user reading and add nocolor flag by default
2022-03-31 03:25:53 +03:00
2bc2b63a8a add help command 2022-03-31 02:18:39 +03:00
28cc38aaa5
port part of settings to database (#54) 2022-03-31 01:48:06 +03:00
fb02e676af migrate to tox
this also requires to move default configuration files to share/ahriman.
Thus the following features have been added
* default configuration is not stored in /usr/share/ahriman/settings
* package installed via PKGBUILD now copies files from /usr
* configuration class now fallbacks to default in /usr
2022-03-21 04:38:42 +03:00
13121298f5 deprecate init/repo-init command
In current workflow you need to run setup to run init (because of
repository name), but you need to run init before setup (because of
repository tree rights).

New solution just add `Repo.init()` method call to setup subcommand
after config reload to make sure that repository name has been applied.
In addition chown method as well as setuid method for check_output have
been added.
2022-03-21 01:32:11 +03:00
57f25c309a add console printer
also add python-requests as explicit dependency and escape symbols in
repository name for badges in default tempate
2022-03-20 02:31:50 +03:00
d33844e7b2 ALLOW_AUTO_ARCHITECTURE_RUN flag for unsafe commands 2022-03-19 03:08:45 +03:00
61efbb71a2
add docker support (#52)
* add docker support

* make shellcheck happy
2022-03-13 23:43:25 +03:00
9f4acacada strict test checks 2022-03-04 02:04:52 +03:00
683abca9e5
use own aur wrapper (#49) 2022-02-06 03:44:57 +03:00
bfca7e41ab handle dependencies recursively 2021-12-22 19:35:09 +03:00
603c5449a8
initial implementation of the local git clones (#48) 2021-12-22 15:56:24 +03:00
042638d40e
handle packages which have been removed from the repository (#45)
* handle packages which have been removed from the repository

* manually remove packages which have been removed from the base
2021-11-10 01:37:25 +03:00
a2639f8dbb add update printer which will print current version if any 2021-10-27 03:11:43 +03:00
d3ea81d234 unify aur.search method
due to specific of the AUR API in order to reduce the code we are using
own wrapper and work with it instead of direct library calls
2021-10-26 04:49:55 +03:00
09b0f2914d Add ability to show more info in search and status subcommands
This feature also introduces the followiing changes
* aur-search command now works as expected with multiterms
* printer classes for managing of data print
* --sort-by argument for aur-search subcommand instead of using package
  name
* --quiet argument now has also --no-quite option
* if --quite is supplied, the log level will be set to warn instead of
  critical to be able to see error messages
* pretty_datetime function now also supports datetime objects
* BuildStatus is now pure dataclass
2021-10-26 04:27:36 +03:00
dfd87c502f split application class into traits 2021-10-23 13:44:57 +03:00
0b9ab09879 add patches to clean command 2021-10-20 03:22:16 +03:00
47c54f0b40 add ability to download package from external links (e.g. HTTP) 2021-10-20 03:09:58 +03:00
22d92e3b4e add repo-status-update subcommand 2021-10-17 06:38:49 +03:00
4521c2adde disallow to create tree in case of unsafe run 2021-10-14 04:53:09 +03:00
5c5e54228f use generic removal method 2021-10-14 04:08:21 +03:00
6514924b2d change method spelling
in order to sort method correctly we are going to use the following
namiing schema:

{subject}_{action}_{details}

This schema still have some exceptions, e.g. single word methods, bool
methods (is_) and getters in case if they are singular (i.e. there is
no any other method with this subject)
2021-10-14 04:01:54 +03:00
4502931c39 exactly one called with instead of last call check 2021-10-14 03:12:45 +03:00
fcb167b1a3
github upload support (#41) 2021-10-14 02:30:13 +03:00
72b26603bf
add ability to add manually stored packages (#40)
* add ability to add manually stored packages

* update tests

* handle manual packages in remove-unknown method

* live fixes

also rename branches to has_remotes method and change return type
2021-10-12 21:15:35 +03:00
7c4f84fbc7 more verbose help messages 2021-10-07 01:56:22 +03:00
3b6b2efcb1 patch control subcommands 2021-10-05 08:57:42 +03:00
9f99dd3ff2
patch support (#35) 2021-10-03 15:20:36 +03:00
6becd01803 replace no-log with quiet
Also behavior of the flag has been changed: now it disables logs at all
2021-10-03 01:59:33 +03:00
59f2992559 do not use set_defaults for architecture arguments
according to the source code defaults always updates the values
dictionary. This in this specific case it is impossible to override the
value it will be always empty list.

In order to handle it we are adding another property to the Handler
class which allows to run with None architecture list.

This particular set_defaults behaviour is still useful for other cases
when we have to run command without any specific architecture
2021-10-03 00:59:24 +03:00
51b28baf40 add ability to specify package source explicitly during the addition 2021-09-26 09:55:14 +03:00
36c763069d only set file rights if requested 2021-09-23 20:36:22 +03:00
16bb1403a1 add ability to reload authentication module 2021-09-17 16:05:38 +03:00
41731ca359 add ability to remove an user
also replace old user by new one before creation
2021-09-16 02:41:56 +03:00
e99c2b0c83 remove own implementations of getlist and getpath method in order to use
converters feature
2021-09-14 03:57:20 +03:00
6294c0ba14 add ability to filter status response by package status 2021-09-13 23:27:36 +03:00
2c74be31bd raise InvalidCommand exception in case if remove option supplied without
package
2021-09-13 22:38:38 +03:00
98eb93c27a
Add ability to trigger updates from the web (#31)
* add external process spawner and update test cases

* pass no_report to handlers

* provide service api endpoints

* do not spawn process for single architecture run

* pass no report to handlers

* make _call method of handlers public and also simplify process spawn

* move update under add

* implement actions from web page

* clear logging & improve l&f
2021-09-10 00:33:35 +03:00
18de70154e add option to set user-password for service when updating its hash 2021-09-05 15:40:03 +03:00
9b8c9b2b2d
migration of jinja tempaltes to bootstrap (#30) 2021-09-05 05:27:58 +03:00