Commit Graph

154 Commits

Author SHA1 Message Date
ded896ee1b gracefully terminate web server
In previous revisions server was terminated by itself, thus no lock or
socket was removed. In new version, graceful termination of the queue
has been added as well as server now handles singals
2023-03-06 01:13:41 +02:00
20974dae6f add more validation rules 2023-02-23 15:18:56 +02:00
aae3c82f3d update dependencies before build (#91)
Old implementation has used add step in order to fetch dependencies,
which could lead to build errors in case if dependency list was updated.

New solution uses dependencies which are declared at current version and
fetch them (if required and if enabled) before update process.

Closes #90
2023-02-12 06:02:30 +03:00
82322f7a6c mask mypy warning
The newest mypy produces the following warning:

src/ahriman/application/handlers/search.py:43: error: Non-overlapping identity check (left operand type: "Union[_DefaultFactory[Any], Literal[_MISSING_TYPE.MISSING]]", right operand type: "Type[List[Any]]")  [comparison-overlap]

which is more likely caused by updated dataclass models to protoocol (however decorators are still calllable). This commit masks problematic line from checking
2023-02-09 22:46:08 +02:00
125da217d3 hide passwords and secrets from repo-config subcommand by default 2023-02-05 16:44:48 +02:00
050041b2de handle architecture specific fields for dependencies
This change requires srcinfo at least 0.1.2 version. Unfortunatelly aur
api don't support architecture specific arrays for now, so we just leave
it as is

Closes #82
2023-01-31 14:34:09 +02:00
c1718b3862 calculate dependencies based on package information (#89) 2023-01-30 17:28:05 +03:00
93876307f8 add ability to override pacman mirror in devtools configuration
This commit also extends configuration of the multilib option, adding
the ability to exlcude multilib repository from repositories list

Note, that in order to support repository list and mirror correctly,
alpm configuration section is now architectture specific
2023-01-15 18:05:15 +02:00
419da3db89 add ability to check for service updates 2023-01-12 15:37:05 +02:00
00ca06ffab subcommand review
Some commands have been moved to another group and thus having another
default name (old subcommands are still available...for now):

* daemon -> repo-daemon
* key-import -> service-key-import
* repo-clean -> service-clean
* repo-config -> service-config
* repo-config-validate -> service-config-validate
* repo-setup -> service-setup
* repo-shell -> service-shell
* version -> help-version
2023-01-12 15:37:05 +02:00
5a05c8ce91 expose trigger configuration schema
Note that this commit contains the following breaking changes:

* remote pull and remote push triggers are now enabled by default (with
  empty target list)
* remote pull and remote push triggers now require target option to be
  set (old behaviour had fallback on `gitremote`)
* validation is now considered to be stable, so it is enabled by default
  in docker image (can be disabled however)
2023-01-10 03:33:52 +02:00
b09aea13af add config validator subcommand (#80)
* add config validator subcommand

* add --exit-code flag

* docs & faq update
2023-01-09 18:22:29 +03:00
e0126bb811 add repo-structure subcommand
This commit also changes Tree class, replacing load method by resolve
2022-12-27 10:35:03 +02:00
83e9d7c523 write patches via gitremote push trigger (#79)
* write patches via gitremote push trigger

* implement context variables intead of custom database class
2022-12-25 02:10:38 +03:00
ebd06cb443 improve setup command by --makeflags-jobs argument and fix repository sign on creation 2022-12-02 01:41:23 +02:00
fe66c6c45c implement support of unix socket for server
This feature can be used for unauthorized access to apis - e.g. for
reporting service if it is run on the same machine. Since now it becomes
recommended way for the interprocess communication, thus some options
(e.g. creating user with as-service flag) are no longer available now
2022-12-02 01:41:23 +02:00
bba58352e0 do not invoke configuration write in case if no salt or user was written 2022-12-02 01:41:23 +02:00
41cc58ed31 add rebuild implementation to interface 2022-12-02 01:41:23 +02:00
9fa1fa108f add key-import button to interface 2022-12-02 01:41:23 +02:00
81fa73f725 ask users to repeat password
In case if password is asked via getpass, it is possible to make typo
and user will not see the mistake. In order to avoid it, additional
confirmation has been added
2022-11-22 02:19:37 +02:00
14cb548c3b Extended package status page (#76)
* implement log storage at backend
* handle process id during removal. During one process we can write logs from different packages in different times (e.g. check and update later) and we would like to store all logs belong to the same process
* set package context in main functions
* implement logs support in interface
* filter out logs posting http logs
* add timestamp to log records
* hide getting logs under reporter permission

List of breaking changes:

* `ahriman.core.lazy_logging.LazyLogging` has been renamed to `ahriman.core.log.LazyLogging`
* `ahriman.core.configuration.Configuration.from_path` does not have `quiet` attribute now
* `ahriman.core.configuration.Configuration` class does not have `load_logging` method now
* `ahriman.core.status.client.Client.load` requires `report` argument now
2022-11-22 02:58:22 +03:00
f817b32976 use intersection of from_database and depends_on filters for the rebuild subcommand
Old logic used OR condition, i.e. if set from-database, it would ignore
the --depends-on flag. In new logic it calculates dependencies based on
the package list, which can be retrieved from database
2022-11-11 16:02:19 +02:00
41080c5ff8 add exec to shell subcommand 2022-11-10 18:52:55 +02:00
b75bd30658 replace several store_true keys to booleanoptionalaction alternative (#74) 2022-11-10 18:34:01 +03:00
b3b3cad706 write .makepkg.conf to home dir instead of repository root (#72) 2022-11-08 17:02:13 +02:00
82b932371b add ability to use ahriman pacman database instead of system one (#71)
By default this feature is enabled. On the first run it will copy (if
exists) databases from filesystem to local cache (one per each
architecture). Later it will use this cache for all alpm operations. In
order to update this cache, some commands (mainly package building)
provide `-y`/`--refresh` option which has same semantics as pacman -Sy
does.

Note however that due to extending `Pacman` class some methods were
renamed in order to be more descriptive:
* `Pacman.all_packages` -> `Pacman.packages`
* `Pacman.get` -> `Pacman.package_get`

This commit also adds multilib repository to the default docker image
which was missed.
2022-11-08 17:26:51 +03:00
45fb2f3c46 improve repo-setup command
* Move devtools executable to ahriman home, because we don't really
  need to use executable inside root
* Use named sudoers file instead of single file. It will allow easily to
  remove file as well as use setup command for several
  repositories/architectures
2022-11-06 14:25:44 +02:00
3d98dd267a fix update handler tests
If daemon tests are run before the update handler it causes test
failure, because there are other calls
2022-10-31 11:51:12 +02:00
8e2732f6fe add daemon subcommand
This command emulates default systemd timer and can be useful in docker
container in order to run 24/7
2022-10-31 01:38:01 +02:00
73e311a41c implement single-function patches (#69) 2022-10-30 03:11:03 +03:00
a5ce6b78dd Add gitremote triggers (#68)
* add gitremote pull trigger

* add push gitremote trigger

* docs update
2022-10-18 01:46:27 +03:00
1a83dd6f5a extend triggers to on_start and on_stop methods
This commit also replaces old run method to new on_result
2022-09-26 01:22:54 +03:00
9d016f51b5 frozen dataclasses 2022-07-26 14:40:28 +03:00
7b647a9b5a fix case with package name which cannot be downloaded
(without special settings)

The issue appears if file or its version contains one of special URI
characters, e.g. +. Theu will be interpreted as query parameters by
(some) servers (e.g. S3 works in this way). In this commit we rename
archive to the one with safe name.
2022-06-27 18:53:48 +03:00
d7966e419d add shell and version parser 2022-05-27 18:34:07 +03:00
b1dfafe275 dynamic html load (#63)
* dynamic html load
* split by classes
2022-05-20 22:29:36 +03:00
375f9fcfb7 add ability to run only speicifed triggers from command line
This commit also restores repo-report and repo-sync subcommands
2022-05-12 18:43:36 +03:00
b9cd98235e triggers implementation (#62) 2022-05-09 20:00:20 +03:00
7f87863734 shorten public imports 2022-05-06 04:08:05 +03:00
571f720ae2 Complete official repository support (#59) 2022-05-03 00:49:32 +03:00
5030395025 rename classes in order to make documentation reference work 2022-04-18 05:30:02 +03:00
4daff81d5f Docstring update (#58)
* migrate docstrings from reST to google format

* add raises note

Also change behaviour of the `from_option` method to fallback to
disabled instead of raising exception on unknown option

* fix part of warnings for sphinx

* make identation a bit more readable

* review fixes

* add verbose description for properties to make them parsed by sphinx extenstion

* add demo sphinx generator
2022-04-17 20:25:28 +03:00
c094d4ee79 add backup and restore subcommands 2022-04-10 21:34:34 +03:00
3cb479ba4b merge restore to rebuild commannd 2022-04-10 01:51:12 +03:00
9f7995fb2f add support of repository restoration 2022-04-07 04:49:07 +03:00
9ce1c36f35 add support of officiall repositories api 2022-04-07 04:19:37 +03:00
4990ce4198 fix descriptions 2022-04-06 01:48:03 +03:00
d5503b22ba add configurable exit codes to some commands (#55) 2022-04-01 18:30:11 +03:00
a132b1544a create user in docker container
also fix user reading and add nocolor flag by default
2022-03-31 03:25:53 +03:00
8e6473d2a0 add help command 2022-03-31 02:18:39 +03:00
83931f5cf4 port part of settings to database (#54) 2022-03-31 01:48:06 +03:00
d4eadf0013 migrate to tox
this also requires to move default configuration files to share/ahriman.
Thus the following features have been added
* default configuration is not stored in /usr/share/ahriman/settings
* package installed via PKGBUILD now copies files from /usr
* configuration class now fallbacks to default in /usr
2022-03-21 04:38:42 +03:00
04174a3e6d deprecate init/repo-init command
In current workflow you need to run setup to run init (because of
repository name), but you need to run init before setup (because of
repository tree rights).

New solution just add `Repo.init()` method call to setup subcommand
after config reload to make sure that repository name has been applied.
In addition chown method as well as setuid method for check_output have
been added.
2022-03-21 01:32:11 +03:00
a7c6d95b34 add console printer
also add python-requests as explicit dependency and escape symbols in
repository name for badges in default tempate
2022-03-20 02:31:50 +03:00
060c7412b1 ALLOW_AUTO_ARCHITECTURE_RUN flag for unsafe commands 2022-03-19 03:08:45 +03:00
046febc440 add docker support (#52)
* add docker support

* make shellcheck happy
2022-03-13 23:43:25 +03:00
9964a96296 strict test checks 2022-03-04 02:04:52 +03:00
9197b416e6 use own aur wrapper (#49) 2022-02-06 03:44:57 +03:00
1e85c1db70 handle dependencies recursively 2021-12-22 19:35:09 +03:00
32aaa4a1f8 initial implementation of the local git clones (#48) 2021-12-22 15:56:24 +03:00
0a8b172e9c unify aur.search method
due to specific of the AUR API in order to reduce the code we are using
own wrapper and work with it instead of direct library calls
2021-10-26 04:49:55 +03:00
7d782f120d Add ability to show more info in search and status subcommands
This feature also introduces the followiing changes
* aur-search command now works as expected with multiterms
* printer classes for managing of data print
* --sort-by argument for aur-search subcommand instead of using package
  name
* --quiet argument now has also --no-quite option
* if --quite is supplied, the log level will be set to warn instead of
  critical to be able to see error messages
* pretty_datetime function now also supports datetime objects
* BuildStatus is now pure dataclass
2021-10-26 04:27:36 +03:00
35df354c14 split application class into traits 2021-10-23 13:44:57 +03:00
4d4e5e9dde add patches to clean command 2021-10-20 03:22:16 +03:00
233b1f7f39 disallow to create tree in case of unsafe run 2021-10-14 04:53:09 +03:00
a83f1d5aa5 use generic removal method 2021-10-14 04:08:21 +03:00
be09c04e78 change method spelling
in order to sort method correctly we are going to use the following
namiing schema:

{subject}_{action}_{details}

This schema still have some exceptions, e.g. single word methods, bool
methods (is_) and getters in case if they are singular (i.e. there is
no any other method with this subject)
2021-10-14 04:01:54 +03:00
86bd49ed1b exactly one called with instead of last call check 2021-10-14 03:12:45 +03:00
2f5790f69f add ability to add manually stored packages (#40)
* add ability to add manually stored packages

* update tests

* handle manual packages in remove-unknown method

* live fixes

also rename branches to has_remotes method and change return type
2021-10-12 21:15:35 +03:00
ef44895b0d more verbose help messages 2021-10-07 01:56:22 +03:00
5cfffbcd46 patch control subcommands 2021-10-05 08:57:42 +03:00
edef4944f6 replace no-log with quiet
Also behavior of the flag has been changed: now it disables logs at all
2021-10-03 01:59:33 +03:00
1192f12f91 do not use set_defaults for architecture arguments
according to the source code defaults always updates the values
dictionary. This in this specific case it is impossible to override the
value it will be always empty list.

In order to handle it we are adding another property to the Handler
class which allows to run with None architecture list.

This particular set_defaults behaviour is still useful for other cases
when we have to run command without any specific architecture
2021-10-03 00:59:24 +03:00
427ba0f0ea add ability to specify package source explicitly during the addition 2021-09-26 09:55:14 +03:00
f333e89bd1 only set file rights if requested 2021-09-23 20:36:22 +03:00
6d1f641e50 add ability to reload authentication module 2021-09-17 16:05:38 +03:00
0cf7756ec4 add ability to remove an user
also replace old user by new one before creation
2021-09-16 02:41:56 +03:00
b561bcc25d remove own implementations of getlist and getpath method in order to use
converters feature
2021-09-14 03:57:20 +03:00
25d76b0482 add ability to filter status response by package status 2021-09-13 23:27:36 +03:00
d2fe4b044f raise InvalidCommand exception in case if remove option supplied without
package
2021-09-13 22:38:38 +03:00
11c03a9041 Add ability to trigger updates from the web (#31)
* add external process spawner and update test cases

* pass no_report to handlers

* provide service api endpoints

* do not spawn process for single architecture run

* pass no report to handlers

* make _call method of handlers public and also simplify process spawn

* move update under add

* implement actions from web page

* clear logging & improve l&f
2021-09-10 00:33:35 +03:00
214f319123 add option to set user-password for service when updating its hash 2021-09-05 15:40:03 +03:00
19ba2ba8e5 migration of jinja tempaltes to bootstrap (#30) 2021-09-05 05:27:58 +03:00
f49894107a add remove uknown method (#29) 2021-09-03 02:28:27 +03:00
b7cff0a19d Auth support (#25)
* initial auth implementation

* add create user parser

* add tests

* update dependencies list

* add login annd logout to index also improve auth

* realworld fixes

* add method set_option to Configuration and also use it everywhere
* split CreateUser handler to additional read method
* check user duplicate on auth mapping read
* generate salt by using passlib instead of random.choice
* case-insensetive usernames
* update dependencies
* update configuration reference
* improve tests

* fix codefactor errors

* hide fields if authorization is enabled, but no auth supplied

* add settings object for auth provider

* readme update
2021-09-02 23:36:00 +03:00
d548fa02f0 add docstrings for every fixture and test methods
also add tests for missing components
2021-08-11 01:55:27 +03:00
33cfa8fef8 Feature/all archs (#21)
* add init subcommand

* add also init command to repository object

* add ability to generate list of architectures

* check if architecture list is not empty
2021-05-23 16:40:40 +03:00
2eb2d3a70a allow to specify list of package dependencies in rebuild target
also replace nargs= by action=append in non-positional args. It is
required to make arguments parsing result more predictable and
consistent
2021-04-18 13:34:27 +03:00
856a94ff00 import pgp key implementation (#17)
* import pgp key implementation

* do not ask confirmation for local sign. Also add argparser test

* superseed requests by python-aur package

* ...and drop --skippgpcheck makgepkg flag by default
2021-04-10 01:37:45 +03:00
f2b39fe439 add search subparser (#15) 2021-04-09 11:57:06 +03:00
fcb892f39f implicit type conversion from command line 2021-04-04 23:53:30 +03:00
1785b0470f more options in setup command 2021-04-04 15:42:06 +03:00
c04f940ad3 fix check errors 2021-04-04 14:00:42 +03:00
207da4caa7 100% coverage 2021-04-03 21:30:57 +03:00
cad85b0f20 add ability to fitler by dependency list 2021-04-02 04:20:39 +03:00
3e8e54857f add ability to remove package from status page 2021-04-02 01:26:46 +03:00
e21b990777 add now argument to add command 2021-03-31 02:19:13 +03:00
cf276f2398 make configuration object arch-specific 2021-03-31 00:04:13 +03:00
4ca2348f0d more verbose variables 2021-03-30 05:29:13 +03:00
e87402fdde add more tests 2021-03-30 01:42:01 +03:00