arcanis/ahriman
1
0
Fork 0
mirror of https://github.com/arcan1s/ahriman.git synced 2025-06-28 06:41:43 +00:00
Code Issues Projects Releases Wiki Activity

docs update

Browse Source
This commit is contained in:
Evgenii Alekseev
2021-10-15 04:12:37 +03:00
parent 233b1f7f39
commit 5f7f58041d
12 changed files with 489 additions and 81 deletions
Download Patch File Download Diff File Expand all files Collapse all files

100
docs/ahriman.1
View File

@ -3,7 +3,7 @@
ahriman
.SH SYNOPSIS
.B ahriman
[-h] [-a ARCHITECTURE] [-c CONFIGURATION] [--force] [-l LOCK] [--no-report] [-q] [--unsafe] [-v] {aur-search,search,key-import,package-add,add,package-remove,remove,package-status,status,package-status-remove,package-status-update,status-update,patch-add,patch-list,patch-remove,repo-check,check,repo-clean,clean,repo-config,config,repo-init,init,repo-rebuild,rebuild,repo-remove-unknown,remove-unknown,repo-report,report,repo-setup,setup,repo-sign,sign,repo-sync,sync,repo-update,update,user-add,user-remove,web} ...
[-h] [-a ARCHITECTURE] [-c CONFIGURATION] [--force] [-l LOCK] [--no-report] [-q] [--unsafe] [-v] {aur-search,search,key-import,package-add,add,package-update,package-remove,remove,package-status,status,package-status-remove,package-status-update,status-update,patch-add,patch-list,patch-remove,repo-check,check,repo-clean,clean,repo-config,config,repo-init,init,repo-rebuild,rebuild,repo-remove-unknown,remove-unknown,repo-report,report,repo-setup,setup,repo-sign,sign,repo-sync,sync,repo-update,update,user-add,user-remove,web} ...
.SH DESCRIPTION
ArcH Linux ReposItory MANager
.SH OPTIONS
@ -34,7 +34,7 @@ force disable any logging
.TP
\fB\-\-unsafe\fR
allow to run ahriman as non\-ahriman user
allow to run ahriman as non\-ahriman user. Some actions might be unavailable
.TP
\fB\-v\fR, \fB\-\-version\fR
@ -65,13 +65,13 @@ remove package status
update package status
.TP
\fBahriman\fR \fI\,patch-add\/\fR
patches control
add patch set
.TP
\fBahriman\fR \fI\,patch-list\/\fR
patches control
list patch sets
.TP
\fBahriman\fR \fI\,patch-remove\/\fR
patches control
remove patch set
.TP
\fBahriman\fR \fI\,repo-check\/\fR
check for updates
@ -107,10 +107,10 @@ sync repository
update packages
.TP
\fBahriman\fR \fI\,user-add\/\fR
create or update user for web services
create or update user
.TP
\fBahriman\fR \fI\,user-remove\/\fR
remove user for web services
remove user
.TP
\fBahriman\fR \fI\,web\/\fR
web server
@ -137,7 +137,7 @@ search terms, can be specified multiple times
.SH OPTIONS 'ahriman key-import'
usage: ahriman key-import [-h] [--key-server KEY_SERVER] key
import PGP key from public sources to repository user
import PGP key from public sources to the repository user
.TP
\fBkey\fR
@ -149,22 +149,22 @@ key server for key import
.SH OPTIONS 'ahriman package-add'
usage: ahriman package-add [-h] [-n]
[-s {PackageSource.Auto,PackageSource.Archive,PackageSource.Directory,PackageSource.AUR}]
[-s {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local}]
[--without-dependencies]
package [package ...]
add package
add existing or new package to the build queue
.TP
\fBpackage\fR
package base/name or archive path
package base/name or path to local files
.TP
\fB\-n\fR, \fB\-\-now\fR
run update function after
.TP
\fB\-s\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.Directory,PackageSource.AUR}, \fB\-\-source\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.Directory,PackageSource.AUR}
\fB\-s\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local}, \fB\-\-source\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local}
package source
.TP
@ -173,22 +173,46 @@ do not add dependencies
.SH OPTIONS 'ahriman add'
usage: ahriman package-add [-h] [-n]
[-s {PackageSource.Auto,PackageSource.Archive,PackageSource.Directory,PackageSource.AUR}]
[-s {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local}]
[--without-dependencies]
package [package ...]
add package
add existing or new package to the build queue
.TP
\fBpackage\fR
package base/name or archive path
package base/name or path to local files
.TP
\fB\-n\fR, \fB\-\-now\fR
run update function after
.TP
\fB\-s\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.Directory,PackageSource.AUR}, \fB\-\-source\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.Directory,PackageSource.AUR}
\fB\-s\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local}, \fB\-\-source\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local}
package source
.TP
\fB\-\-without\-dependencies\fR
do not add dependencies
.SH OPTIONS 'ahriman package-update'
usage: ahriman package-add [-h] [-n]
[-s {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local}]
[--without-dependencies]
package [package ...]
add existing or new package to the build queue
.TP
\fBpackage\fR
package base/name or path to local files
.TP
\fB\-n\fR, \fB\-\-now\fR
run update function after
.TP
\fB\-s\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local}, \fB\-\-source\fR {PackageSource.Auto,PackageSource.Archive,PackageSource.AUR,PackageSource.Directory,PackageSource.Local}
package source
.TP
@ -198,7 +222,7 @@ do not add dependencies
.SH OPTIONS 'ahriman package-remove'
usage: ahriman package-remove [-h] package [package ...]
remove package
remove package from the repository
.TP
\fBpackage\fR
@ -208,7 +232,7 @@ package name or base
.SH OPTIONS 'ahriman remove'
usage: ahriman package-remove [-h] package [package ...]
remove package
remove package from the repository
.TP
\fBpackage\fR
@ -296,7 +320,7 @@ new status
.SH OPTIONS 'ahriman patch-add'
usage: ahriman patch-add [-h] [-t TRACK] package
create/update for sources
create or update source patches
.TP
\fBpackage\fR
@ -329,7 +353,7 @@ package base
.SH OPTIONS 'ahriman repo-check'
usage: ahriman repo-check [-h] [--no-vcs] [package ...]
check for updates. Same as update \-\-dry\-run \-\-no\-manual
check for packages updates. Same as update \-\-dry\-run \-\-no\-manual
.TP
\fBpackage\fR
@ -342,7 +366,7 @@ do not check VCS packages
.SH OPTIONS 'ahriman check'
usage: ahriman repo-check [-h] [--no-vcs] [package ...]
check for updates. Same as update \-\-dry\-run \-\-no\-manual
check for packages updates. Same as update \-\-dry\-run \-\-no\-manual
.TP
\fBpackage\fR
@ -355,7 +379,7 @@ do not check VCS packages
.SH OPTIONS 'ahriman repo-clean'
usage: ahriman repo-clean [-h] [--no-build] [--no-cache] [--no-chroot] [--no-manual] [--no-packages]
clear local caches
remove local caches
.TP
@ -381,7 +405,7 @@ do not clear directory with built packages
.SH OPTIONS 'ahriman clean'
usage: ahriman repo-clean [-h] [--no-build] [--no-cache] [--no-chroot] [--no-manual] [--no-packages]
clear local caches
remove local caches
.TP
@ -407,14 +431,14 @@ do not clear directory with built packages
.SH OPTIONS 'ahriman repo-config'
usage: ahriman repo-config [-h]
dump configuration for specified architecture
dump configuration for the specified architecture
.SH OPTIONS 'ahriman config'
usage: ahriman repo-config [-h]
dump configuration for specified architecture
dump configuration for the specified architecture
@ -435,7 +459,7 @@ create empty repository tree. Optional command for auto architecture support
.SH OPTIONS 'ahriman repo-rebuild'
usage: ahriman repo-rebuild [-h] [--depends-on DEPENDS_ON]
rebuild whole repository
force rebuild whole repository
.TP
@ -445,7 +469,7 @@ only rebuild packages that depend on specified package
.SH OPTIONS 'ahriman rebuild'
usage: ahriman repo-rebuild [-h] [--depends-on DEPENDS_ON]
rebuild whole repository
force rebuild whole repository
.TP
@ -455,7 +479,7 @@ only rebuild packages that depend on specified package
.SH OPTIONS 'ahriman repo-remove-unknown'
usage: ahriman repo-remove-unknown [-h] [--dry-run]
remove packages which are missing in AUR
remove packages which are missing in AUR and do not have local PKGBUILDs
.TP
@ -465,7 +489,7 @@ just perform check for packages without removal
.SH OPTIONS 'ahriman remove-unknown'
usage: ahriman repo-remove-unknown [-h] [--dry-run]
remove packages which are missing in AUR
remove packages which are missing in AUR and do not have local PKGBUILDs
.TP
@ -475,7 +499,7 @@ just perform check for packages without removal
.SH OPTIONS 'ahriman repo-report'
usage: ahriman repo-report [-h] [target ...]
generate report
generate repository report according to current settings
.TP
\fBtarget\fR
@ -485,7 +509,7 @@ target to generate report
.SH OPTIONS 'ahriman report'
usage: ahriman repo-report [-h] [target ...]
generate report
generate repository report according to current settings
.TP
\fBtarget\fR
@ -575,7 +599,7 @@ port of the web service
.SH OPTIONS 'ahriman repo-sign'
usage: ahriman repo-sign [-h] [package ...]
(re\-)sign packages and repository database
(re\-)sign packages and repository database according to current settings
.TP
\fBpackage\fR
@ -585,7 +609,7 @@ sign only specified packages
.SH OPTIONS 'ahriman sign'
usage: ahriman repo-sign [-h] [package ...]
(re\-)sign packages and repository database
(re\-)sign packages and repository database according to current settings
.TP
\fBpackage\fR
@ -595,7 +619,7 @@ sign only specified packages
.SH OPTIONS 'ahriman repo-sync'
usage: ahriman repo-sync [-h] [target ...]
sync packages to remote server
sync repository files to remote server according to current settings
.TP
\fBtarget\fR
@ -605,7 +629,7 @@ target to sync
.SH OPTIONS 'ahriman sync'
usage: ahriman repo-sync [-h] [target ...]
sync packages to remote server
sync repository files to remote server according to current settings
.TP
\fBtarget\fR
@ -615,7 +639,7 @@ target to sync
.SH OPTIONS 'ahriman repo-update'
usage: ahriman repo-update [-h] [--dry-run] [--no-aur] [--no-manual] [--no-vcs] [package ...]
run updates
check for packages updates and run build process if requested
.TP
\fBpackage\fR
@ -640,7 +664,7 @@ do not check VCS packages
.SH OPTIONS 'ahriman update'
usage: ahriman repo-update [-h] [--dry-run] [--no-aur] [--no-manual] [--no-vcs] [package ...]
run updates
check for packages updates and run build process if requested
.TP
\fBpackage\fR
@ -697,7 +721,7 @@ set file permissions to user\-only
.SH OPTIONS 'ahriman user-remove'
usage: ahriman user-remove [-h] [--no-reload] [-s] username
remove user from the user mapping and write the configuration
remove user from the user mapping and update the configuration
.TP
\fBusername\fR

10
docs/architecture.md
View File

@ -133,6 +133,16 @@ OAuth's implementation also allows authenticating users via username + password
In order to configure users there is special command.
## Remote synchronization
There are several supported synchronization providers, currently they are `rsync`, `s3`, `github`.
`rsync` provider does not have any specific logic except for running external rsync application with configured arguments. The service does not handle SSH configuration, thus it has to be configured before running application manually.
`s3` provider uses `boto3` package and implements sync feature. The files are stored in architecture directory (e.g. if bucket is `repository`, packages will be stored in `repository/x86_64` for the `x86_64` architecture), bucket must be created before any action and API key must have permissions to write to the bucket. No external configuration required. In order to upload only changed files the service compares calculated hashes with the Amazon ETags, used realization is described [here](https://teppen.io/2018/10/23/aws_s3_verify_etags/).
`github` provider authenticates through basic auth, API key with repository write permissions is required. There will be created a release with the name of the architecture in case if it does not exist; files will be uploaded to the release assets. It also stores array of files and their MD5 checksums in release body in order to upload only changed ones. According to the Github API in case if there is already uploaded asset with the same name (e.g. database files), asset will be removed first.
## Additional features
Some features require optional dependencies to be installed:

2
docs/configuration.md
View File

@ -121,7 +121,7 @@ Group name must refer to architecture, e.g. it should be `github:x86_64` for x86
Group name must refer to architecture, e.g. it should be `rsync:x86_64` for x86_64 architecture. Requires `rsync` package to be installed. Do not forget to configure ssh for user `ahriman`.
* `command` - rsync command to run, space separated list of string, required.
* `remote` - remote server to rsync (e.g. `1.2.3.4:5678:path/to/sync`), string, required.
* `remote` - remote server to rsync (e.g. `1.2.3.4:path/to/sync`), string, required.
### `s3:*` groups

372
docs/faq.md Normal file
View File

@ -0,0 +1,372 @@
# FAQ
## General topics
### What is the purpose of the project?
This project has been created in order to maintain self-hosted Arch Linux user repository without manual intervention - checking for updates and building packages.
### How do I install it?
TL;DR
```shell
yay -S ahriman
sudo -u ahriman ahriman -a x86_64 init
sudo ahriman -a x86_64 repo-setup --packager "ahriman bot <ahriman@example.com>" --repository "repository"
systemctl enable --now ahriman@x86_64.timer
```
#### Long answer
The idea is to install the package as usual, create working directory tree, create configuration for `sudo` and `devtools`. Detailed description of the setup instruction can be found [here](setup.md).
### Okay, I've installed ahriman, how do I add new package?
```shell
sudo -u ahriman ahriman package-add ahriman --now
```
`--now` flag is totally optional and just run `repo-update` subcommand after the registering the new package, Thus the extended flow is the following:
```shell
sudo -u ahriman ahriman package-add ahriman
sudo -u ahriman ahriman repo-update
```
### AUR is fine, but I would like to create package from local PKGBUILD
TL;DR
```shell
sudo -u ahriman ahriman package-add /path/to/local/directory/with/PKGBUILD --now
```
Before using this command you will need to create local directory, put `PKGBUILD` there and generate `.SRCINFO` by using `makepkg --printsrcinfo > .SRCINFO` command. These packages will be stored locally and _will be ignored_ during automatic update; in order to update the package you will need to run `package-add` command again.
### But I just wanted to change PKGBUILD from AUR a bit!
Well it is supported also.
1. Clone sources from AUR.
2. Make changes you would like to (e.g. edit `PKGBUILD`, add external patches).
3. Run `sudo -u ahriman ahriman patch-add /path/to/local/directory/with/PKGBUILD`.
The last command will calculate diff from current tree to the `HEAD` and will store it locally. Patches will be applied on any package actions (e.g. it can be used for dependency management).
### Package build fails because it cannot validate PGP signature of source files
TL;DR
```shell
sudo -u ahriman ahriman key-import ...
```
### How do I check if there are new commits for VCS packages?
Normally the service handles VCS packages correctly, but it requires additional dependencies:
```shell
pacman -S breezy darcs mercurial subversion
```
### I would like to remove package because it is no longer needed/moved to official repositories
```shell
sudo -u ahriman ahriman package-remove ahriman
```
Also, there is command `repo-remove-unknown` which checks packages in AUR and local storage and removes ones which have been removed.
Remove commands also remove any package files (patches, caches etc).
### There is new major release of %library-name%, how do I rebuild packages?
TL;DR
```shell
sudo -u ahriman ahriman repo-rebuild --depends-on python
```
You can even rebuild the whole repository (which is particular useful in case if you would like to change packager) if you do not supply `--depends-on` option.
However, note that you do not need to rebuild repository in case if you just changed signing option, just use `repo-sign` command instead.
### Hmm, I have packages built, but how can I use it?
Add the following lines to your `pacman.conf`:
```ini
[repository]
Server = file:///var/lib/ahriman/repository/x86_64
```
(You might need to add `SigLevel` option according to the pacman documentation.)
## Remote synchronization
### Wait I would like to use the repository from another server
There are several choices:
1. Easy and cheap, just share your local files through the internet, e.g. for `nginx`:
```
server {
location /x86_64 {
root /var/lib/ahriman/repository/x86_64;
autoindex on;
}
}
```
2. You can also upload your packages using `rsync` to any available server. In order to use it you would need to configure ahriman first:
```ini
[upload]
target = rsync
[rsync]
remote = 192.168.0.1:/srv/repo
```
After that just add `/srv/repo` to the `pacman.conf` as usual. You can also upload to S3 (e.g. `Server = https://s3.eu-central-1.amazonaws.com/repository/x86_64`) or to Github (e.g. `Server = https://github.com/ahriman/repository/releases/download/x86_64`).
### How do I configure S3?
1. Install dependencies:
```shell
pacman -S python-boto3
```
3. Create a bucket.
4. Create user with write access to the bucket:
```
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ListObjectsInBucket",
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::repository"
]
},
{
"Sid": "AllObjectActions",
"Effect": "Allow",
"Action": "s3:*Object",
"Resource": [
"arn:aws:s3:::repository/*"
]
}
]
}
```
5. Create an API key for the user and store it.
6. Configure the service as following:
```ini
[upload]
target = s3
[s3]
access_key = ...
bucket = repository
region = eu-central-1
secret_key = ...
```
### How do I configure Github?
1. Create a repository.
2. [Create API key](https://github.com/settings/tokens) with scope `public_repo`.
3. Configure the service as following:
```ini
[upload]
target = github
[github]
api_key = ...
owner = ahriman
repository = repository
```
## Reporting
### I would like to get report to email
1. Install dependencies:
```shell
yay -S python-jinja
```
2. Configure the service:
```ini
[report]
target = email
[email]
host = smtp.example.com
link_path = http://example.com/x86_64
password = ...
port = 465
receivers = me@example.com
sender = me@example.com
user = me@example.com
```
### I'm using synchronization to S3 and would like to generate index page
1. Install dependencies:
```shell
yay -S python-jinja
```
2. Configure the service:
```ini
[report]
target = html
[html]
path = /var/lib/ahriman/repository/x86_64/index.html
link_path = http://example.com/x86_64
```
After these steps `index.html` file will be automatically synced to S3
## Web service
### Readme mentions web interface, how do I use it?
1. Install dependencies:
```shell
yay -S python-aiohttp python-aiohttp-jinja2
```
2. Configure service:
```ini
[web]
port = 8080
```
3. Start the web service `systemctl enable --now ahriman-web@x86_64`.
### I would like to limit user access to the status page
1. Install dependencies 😊:
```shell
yay -S python-aiohttp-security python-aiohttp-session python-cryptography
```
2. Configure the service to enable authorization:
```ini
[auth]
target = configuration
```
3. Create user for the service:
```shell
sudo -u ahriman ahriman user-add --as-service -r write api
```
This command will ask for the password, just type it in stdin; _do not_ leave the field blank, user will not be able to authorize.
4. Create end-user `sudo -u ahriman ahriman user-add -r write my-first-user` with password.
5. Restart web service `systemctl restart ahriman-web@x86_64`.
### I would like to use OAuth
1. Create OAuth web application, download its `client_id` and `client_secret`.
2. Guess what? Install dependencies:
```shell
yay -S python-aiohttp-security python-aiohttp-session python-cryptography python-aioauth-client
```
3. Configure the service:
```ini
[auth]
target = oauth
client_id = ...
client_secret = ...
[web]
address = https://example.com
```
Configure `oauth_provider` and `oauth_scopes` in case if you would like to use different from Google provider. Scope must grant access to user email. `web.address` is required to make callback URL available from internet.
4. Create service user:
```shell
sudo -u ahriman ahriman user-add --as-service -r write api
```
5. Create end-user `sudo -u ahriman ahriman user-add -r write my-first-user`. When it will ask for the password leave it blank.
6. Restart web service `systemctl restart ahriman-web@x86_64`.
## Other topics
### How does it differ from %another-manager%?
Short answer - I do not know.
#### [archrepo2](https://github.com/lilydjwg/archrepo2)
Don't know, haven't tried it. But it lacks of documentation at least.
* Web interface.
* No synchronization and reporting.
* `archrepo2` actively uses direct shell calls and `yaourt` components.
* It has constantly running process instead of timer process (it is not pro or con).
#### [repo-scripts](https://github.com/arcan1s/repo-scripts)
Though originally I've created ahriman by trying to improve the project, it still lacks a lot of features:
* Web interface.
* Better reporting with template support.
* Synchronization features (there was only `rsync` based).
* Local packages and patches support.
* No dependency management.
* And so on.
`repo-scripts` also have bad architecture and bad quality code and uses out-of-dated `yaourt` and `package-query`.
### I would like to check service logs
By default, the service writes logs to `/dev/log` which can be accessed by using `journalctl` command (logs are written to the journal of the user under which command is run).
You can also edit configuration and forward logs to `stderr`, just change `handlers` value, e.g.:
```shell
sed -i 's/handlers = syslog_handler/handlers = console_handler/g' /etc/ahriman.ini.d/logging.ini
```
You can even configure logging as you wish, but kindly refer to python `logging` module configuration.
### Html customization
It is possible to customize html templates. In order to do so, create files somewhere (refer to Jinja2 documentation and the service source code for available parameters) and put `template_path` to configuration pointing to this directory.
### I did not find my question
[Create an issue](https://github.com/arcan1s/ahriman/issues) with type **Question**.

2
docs/setup.md
View File

@ -59,7 +59,7 @@
6. Add packages by using `ahriman package-add {package}` command:
```shell
sudo -u ahriman ahriman -a x86_64 package-add yay --now
sudo -u ahriman ahriman -a x86_64 package-add ahriman --now
```
## User creation