DR
Existe alguma configuração para o rsnapshot (veja as versões abaixo) que me permitiria restringir a saída aos comandos do shell, erros e notificações de início/término geradas pelo próprio rsnapshot, enquanto fazia com que o rsync gerasse e registrasse o nível de detalhe desejadoapenasno arquivo de log rsync? Ou, de forma mais sucinta, posso fazer com que os resultados da saída do rsnapshot correspondam às descrições de verbosidade no arquivo de configuração do rsnapshot?
Caso contrário, existe uma comunidade rsnapshot que aceita solicitações de recursos?
Só a parte do TL...
Após algumas soluções de problemas (veja abaixo), parece que minha combinação específica de rsnapshot e rsync não funciona mais como nos anos anteriores. Especificamente, a saída do rsync agora aparece na saída do console do rsnapshot, independentemente das configurações detalhadas no rsnapshot.conf. Eu tenho uma nova instalação do FreeBSD 12.2
freebsd-version
12.2-RELEASE-p10
rsync foi instalado como parte pkg install rsnapshote rsync -Vmostra
rsync version 3.2.3 protocol version 31
Copyright (C) 1996-2020 by Andrew Tridgell, Wayne Davison, and others.
Web site: https://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, hardlink-specials, symlinks, IPv6, atimes,
batchfiles, inplace, append, ACLs, xattrs, optional protect-args, iconv,
symtimes, no prealloc, stop-at, no crtimes, file-flags
Optimizations:
no SIMD, no asm, openssl-crypto
Checksum list:
xxh128 xxh3 xxh64 (xxhash) md5 md4 none
Compress list:
zstd lz4 zlibx zlib none
a versão atual do rsnapshot é:
my $VERSION = '1.4.4';
O problema, depois de postar a forma original desta pergunta e dormir sobre ela, além da solução de problemas mencionada acima, se resume a isto: o rsnapshot é executado no crontab e as informações desnecessárias do rsync aparecem na saída do console (apesar da configuração detalhada 1) parece exigir um script "wrapper" para suprimir o ruído e, em seguida, reunir as informações corretas do log do rsnapshot. Este parece ser um processo muito sujeito a erros e que provavelmente exigirá muita manutenção, já que as atualizações quebram a natureza da fita adesiva da solução alternativa.
O caminho de menor custo parece ser desistir de obter informações estatísticas sobre a eficácia da transferência rsync. Isso me permite manter a legibilidade para monitorar o sucesso/falha do rsync e executá-lo novamente quando ele falhar.
Claramente algo mudou, suspeito com o rsync (uma configuração -v implícita?), E se houver alguma configuração que me permita obter o que desejo na saída do rsnapshot (apenas os comandos do shell, erros e notificações de início/término) enquanto gravando a saída do rsync no arquivo de log do rsync, adoraria saber sobre isso.
Por favor, tenha em mente que o regime rsnapshot está funcionando perfeitamente desde 2018, e apenas uma nova instalação do FreeBSD 12.2 o quebrou. O regime rsnapshot anterior estava rodando (no mesmo hardware) em um FreeBSD 12.1 atualizado (iniciado como 10.x). rsync e rsnapshot foram originalmente construídos separadamente (nessa ordem) das portas do FreeBSD 10.x, e atualizados regularmente desde então com o portmaster.
Desta vez (como mencionado), instalei o rsnapshot com o pkg e deixei-o instalar o rsync (e tudo o mais necessário).
rsnapshot.conf mudou apenas no valor snapshot_roote em uma lista reduzida de pontos de backup.
Aqui está a saída que desejo (fácil de processar) para os e-mails de monitoramento (ou seja, sem todo o ruído do rsync):
Wed Oct 20 21:40:00 PDT 2021
=================================================================
echo 98875 > /var/run/rsnapshot.pid
mv /obo-offsitepool/archives/daily.5/ /obo-offsitepool/archives/daily.6/
mv /obo-offsitepool/archives/daily.4/ /obo-offsitepool/archives/daily.5/
mv /obo-offsitepool/archives/daily.3/ /obo-offsitepool/archives/daily.4/
mv /obo-offsitepool/archives/daily.2/ /obo-offsitepool/archives/daily.3/
mv /obo-offsitepool/archives/daily.1/ /obo-offsitepool/archives/daily.2/
native_cp_al("/obo-offsitepool/archives/daily.0", \
"/obo-offsitepool/archives/daily.1")
/usr/local/bin/rsync -a --delete --numeric-ids \
/obo-offsitepool/archives/daily.0/ /obo-offsitepool/archives/daily.1/
/usr/local/bin/rsync -rltv --chmod D0770,F0660 --delete --relative \
--delete-excluded --partial --stats --log-file=/var/log/rsync \
--human-readable \
--exclude-from=/obo-offsitepool/archives/.rsnapshot_excludes \
/usr/local/etc/ \
/obo-offsitepool/archives/daily.0/obo-offsite1/local_etc/
/usr/local/bin/rsync -rltv --chmod D0770,F0660 --delete --relative \
--delete-excluded --partial --stats --log-file=/var/log/rsync \
--human-readable \
--exclude-from=/obo-offsitepool/archives/.rsnapshot_excludes \
192.168.18.3::srv330-group/ \
/obo-offsitepool/archives/daily.0/CSO/srv330-group
... etc
Em vez disso (tendo em mente que existem mais de 20 pontos de backup, muitos com centenas de linhas de saída rsync indesejada):
Tue Oct 26 18:55:00 PDT 2021
=================================================================
echo 97810 > /var/run/rsnapshot.pid
/bin/rm -rf /obopool/archives/daily.6/
mv /obopool/archives/daily.5/ /obopool/archives/daily.6/
mv /obopool/archives/daily.4/ /obopool/archives/daily.5/
mv /obopool/archives/daily.3/ /obopool/archives/daily.4/
mv /obopool/archives/daily.2/ /obopool/archives/daily.3/
mv /obopool/archives/daily.1/ /obopool/archives/daily.2/
native_cp_al("/obopool/archives/daily.0", \
"/obopool/archives/daily.1")
/usr/local/bin/rsync -a --delete --numeric-ids \
/obopool/archives/daily.0/ /obopool/archives/daily.1/
/usr/local/bin/rsync -rltv --chmod D0770,F0660 --delete --relative \
--delete-excluded --partial --stats --log-file=/var/log/rsync \
--human-readable \
--exclude-from=/obopool/archives/.rsnapshot_excludes /usr/local/etc/ \
/obopool/archives/daily.0/offsite1/local_etc/
sending incremental file list
/usr/
deleting usr/local/etc/ssmtp/ssmtp.conf.sample
deleting usr/local/etc/ssmtp/ssmtp.conf.2021-01-20
deleting usr/local/etc/ssmtp/ssmtp.conf
deleting usr/local/etc/ssmtp/revaliases.sample
deleting usr/local/etc/ssmtp/revaliases
deleting usr/local/etc/ssmtp/
deleting usr/local/etc/dma/dma.conf.sample
deleting usr/local/etc/dma/dma.conf
deleting usr/local/etc/dma/auth.conf.sample
deleting usr/local/etc/dma/auth.conf
deleting usr/local/etc/dma/
deleting usr/local/etc/portmaster.rc.sample
deleting usr/local/etc/papersize.letter
deleting usr/local/etc/papersize.a4
deleting usr/local/etc/bash_completion.d/portmaster.sh
deleting usr/local/etc/rc.d/dma_flushq
/usr/local/
/usr/local/etc/
/usr/local/etc/rsnapshot.conf
/usr/local/etc/rsnapshot.conf.default
/usr/local/etc/rsnapshot.conf_2018-09-08
/usr/local/etc/rsnapshot.conf_2019-06-14
/usr/local/etc/rsnapshot.conf_2019-08-23
/usr/local/etc/rsnapshot.conf_2021-02-01
/usr/local/etc/rsnapshot.conf_2021-06-21
/usr/local/etc/screenrc
/usr/local/etc/screenrc.sample
/usr/local/etc/smartd.conf
/usr/local/etc/smartd.conf.sample
/usr/local/etc/smartd_warning.sh
/usr/local/etc/bash_completion.d/
/usr/local/etc/man.d/
/usr/local/etc/man.d/perl5.conf
/usr/local/etc/newsyslog.conf.d/
/usr/local/etc/newsyslog.conf.d/rsnapshot
/usr/local/etc/newsyslog.conf.d/rsync
/usr/local/etc/newsyslog.conf.d/rsyncd
/usr/local/etc/periodic/
/usr/local/etc/periodic/daily/
/usr/local/etc/periodic/daily/smart
/usr/local/etc/periodic/security/
/usr/local/etc/periodic/weekly/
/usr/local/etc/rc.d/
/usr/local/etc/rc.d/rsyncd
/usr/local/etc/rc.d/smartd
/usr/local/etc/rc.d/uuidd
/usr/local/etc/rsync/
/usr/local/etc/rsync/rsyncd.conf
/usr/local/etc/rsync/rsyncd.conf.sample
/usr/local/etc/smartd_warning.d/
/usr/local/etc/ssl/
/usr/local/etc/ssl/cert.pem
Number of files: 46 (reg: 31, dir: 14, link: 1)
Number of created files: 4 (reg: 3, dir: 1)
Number of deleted files: 16 (reg: 14, dir: 2)
Number of regular files transferred: 23
Total file size: 774.49K bytes
Total transferred file size: 735.92K bytes
Literal data: 735.92K bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 738.45K
Total bytes received: 1.10K
sent 738.45K bytes received 1.10K bytes 1.48M bytes/sec
total size is 774.49K speedup is 1.05
/usr/local/bin/rsync -rltv --chmod D0770,F0660 --delete --relative \
--delete-excluded --partial --stats --log-file=/var/log/rsync \
--human-readable \
--exclude-from=/obopool/archives/.rsnapshot_excludes \
192.168.18.3::srv330-group/ \
/obopool/archives/daily.0/CSO/srv330-group
receiving incremental file list
CSO2/Lori/
CSO2/Lori/retail.prices2.docx
Retail/
...etc
Ambos os relatórios de e-mail acima foram produzidos a partir do mesmo arquivo rsnapshot.conf.
Aqui está a solução de problemas que estou fazendo. O teste 4 é a melhor aproximação do que eu tinha anteriormente.
1
rsnapshot -v alpha && /root/scripts/marklogs.sh --> using default rsnapshot.conf with
entries for rsync_short_args AND rsync_long_args were actually elided from file (not left with # symbol)
- normal screen output (start notification, shell commands, success notification)
- rsnapshot log shows same as monitor output, no rsync log output
2 Explicitly add in default settings to the rsnapshot.conf file
rm -rf /obopool/tester/alpha* && rsnapshot -v alpha && /root/scripts/marklogs.sh #--> rsnapshot.conf modifed as shown:
rsync_short_args -a
rsync_long_args --delete --numeric-ids --relative --delete-excluded
- same as previous
3 change rsync_long_args
rm -rf /obopool/tester/alpha* && rsnapshot -v alpha && /root/scripts/marklogs.sh #--> rsnapshot.conf modifed from default as shown:
rsync_short_args -a
rsync_long_args --chmod D0770,F0660 --delete --relative --delete-excluded --partial --log-file=/var/log/rsync-test
- console shows expected output (start notification, shell commands, success confirmations)
- rsnapshot log shows same as monitor output
- rsync log shows building file list, files xferred, sent/received/total summary line for each backup point
4 change rsync_short args
rm -rf /obopool/tester/alpha* && rsnapshot -v alpha && /root/scripts/marklogs.sh #--> rsnapshot.conf modifed from default as shown:
rsync_short_args -rlt
rsync_long_args --chmod D0770,F0660 --delete --relative --delete-excluded --partial --log-file=/var/log/rsync-test
- normal console output (start notification, shell commands, success notification)
- rsnapshot log shows same as console
- rsync log shows building file list, files xferred, sent/received/total summary line for each backup point
5 change rsync_short_arg to add -v
rm -rf /obopool/tester/alpha* && rsnapshot -v alpha && /root/scripts/marklogs.sh #--> rsnapshot.conf modifed from default as shown:
rsync_short_args -rltv
rsync_long_args --chmod D0770,F0660 --delete --relative --delete-excluded --partial --log-file=/var/log/rsync-test
- console shows the unexpected (and undesired), given verbose set at 2, rsync output
- rsnapshog log is polluted with rsync output, shows the start notification & shell commands before the noise, then some closing shell commands (touch /obopool/tester/alpha.0/; rm -f /var/run/rsnapshot.pid) then the completion notification
- rsync log adds one line, "total size ..... speedup is .....", to logged info.
6 change loglevel to 1 to see effect on rsnapshot log
rm -rf /obopool/tester/alpha* && rsnapshot -v alpha && /root/scripts/marklogs.sh #--> rsnapshot.conf modifed from default as shown:
loglevel 1
rsync_short_args -rltv
rsync_long_args --chmod D0770,F0660 --delete --relative --delete-excluded --partial --log-file=/var/log/rsync-test
- console shows same polluted output (rsnapshot shell commands with rsync data)
- rsnapshot log received no ouput
- rsync log shows same (correct) output as previous execution
7 change verbose to 1 (quiet) in rsnapshot.conf
rm -rf /obopool/tester/alpha* && rsnapshot -v alpha && /root/scripts/marklogs.sh #--> rsnapshot.conf modifed as shown:
verbose 1
loglevel 1
rsync_short_args -rltv
rsync_long_args --chmod D0770,F0660 --delete --relative --delete-excluded --partial --log-file=/var/log/rsync-test
- console - no change to output from test 6
- rsnapshot log received no ouput (expected with loglevel 1)
- rsync log shows expected output (same as previous)
8 set loglevel to 2 to verify effect on rsnapshot log
rm -rf /obopool/tester/alpha* && rsnapshot -v alpha && /root/scripts/marklogs.sh #--> rsnapshot.conf modifed as shown:
verbose 1
loglevel 2
rsync_short_args -rltv
rsync_long_args --chmod D0770,F0660 --delete --relative --delete-excluded --partial --log-file=/var/log/rsync-test
- console - no change
- rshapshot log shows only start and completion notices (no shell commands). One assumes errors would show up.
- rsync log shows expected output
9 variation on 4: default verbose/log level - no -v on rsync_short_args, and add --stats to rsync long args
rm -rf /obopool/tester/alpha* && rsnapshot -v alpha && /root/scripts/marklogs.sh #--> rsnapshot.conf modifed from default as shown:
rsync_short_args -rlt
rsync_long_args --chmod D0770,F0660 --delete --relative --delete-excluded --partial --log-file=/var/log/rsync-test --stats
- console has stats (rsync) information mixed in with correct output (start notification, shell commands, success confirmations)
- rshapshot log has the same issues as console: stats output mixed in with expected output (start notification, shell commands, success confirmations)
- rsync log shows expected output -- files plus stats.
Este é o arquivo padrão (instalado) /usr/local/etc/rsnapshot.conf, no qual todas as alterações registradas foram feitas.
#/usr/local/etc/rsnapshot.conf
#################################################
# rsnapshot.conf - rsnapshot configuration file #
#################################################
# #
# PLEASE BE AWARE OF THE FOLLOWING RULE: #
# #
# This file requires tabs between elements #
# #
#################################################
#######################
# CONFIG FILE VERSION #
#######################
config_version 1.2
snapshot_root /obopool/tester
no_create_root 1
cmd_rm /bin/rm
cmd_rsync /usr/local/bin/rsync
cmd_logger /usr/bin/logger
#cmd_rsnapshot_diff /usr/local/bin/rsnapshot-diff
retain alpha 6
retain beta 7
retain gamma 4
#retain delta 3
# Verbose level, 1 through 5.
# 1 Quiet Print fatal errors only
# 2 Default Print errors and warnings only
# 3 Verbose Show equivalent shell commands being executed
# 4 Extra Verbose Show extra verbose information
# 5 Debug mode Everything
#
verbose 2
# Same as "verbose" above, but controls the amount of data sent to the
# logfile, if one is being used. The default is 3.
#
loglevel 3
logfile /var/log/rsnapshot-test
lockfile /var/run/rsnapshot.pid
###############################
### BACKUP POINTS / SCRIPTS ###
###############################
# LOCALHOST
backup /root/ localhost/
backup /etc/ localhost/
backup /usr/local/ localhost/
backup /var/log/ localhost/
e finalmente, aqui está um pouco do perl -Vresultado
perl -V
Summary of my perl5 (revision 5 version 32 subversion 1) configuration:
Platform:
osname=freebsd
osvers=12.2-release-p10
archname=amd64-freebsd-thread-multi
uname='freebsd 122amd64-quarterly-job-03 12.2-release-p10 freebsd 12.2-release-p10 amd64 '
config_args='-Darchlib=/usr/local/lib/perl5/5.32/mach -Dcc=cc -Dcf_by=mat [email protected] -Dcf_time=Sat Jan 23 14:56:40 UTC 2021 -Dinc_version_list=none -Dlibperl=libperl.so.5.32.1 -Dman1dir=/usr/local/lib/perl5/5.32/perl/man/man1 -Dman3dir=/usr/local/lib/perl5/5.32/perl/man/man3 -Dprefix=/usr/local -Dprivlib=/usr/local/lib/perl5/5.32 -Dscriptdir=/usr/local/bin -Dsitearch=/usr/local/lib/perl5/site_perl/mach/5.32 -Dsitelib=/usr/local/lib/perl5/site_perl -Dsiteman1dir=/usr/local/lib/perl5/site_perl/man/man1 -Dsiteman3dir=/usr/local/lib/perl5/site_perl/man/man3 -Dusenm=n -Duseshrplib -sde -Ui_iconv -Ui_malloc -Uinstallusrbinperl -Accflags=-DUSE_THREAD_SAFE_LOCALE -Alddlflags=-L/wrkdirs/usr/ports/lang/perl5.32/work/perl-5.32.1 -L/usr/local/lib/perl5/5.32/mach/CORE -lperl -Dshrpldflags=$(LDDLFLAGS:N-L/wrkdirs/usr/ports/lang/perl5.32/work/perl-5.32.1:N-L/usr/local/lib/perl5/5.32/mach/CORE:N-lperl) -Wl,-soname,$(LIBPERL:R) -Doptimize=-O2 -pipe -fstack-protector-strong -fno-strict-aliasing -Dusedtrace -Ui_gdbm -Dusemultiplicity=y -Duse64bitint -Dusemymalloc=n -Dusethreads=y'
Responder1
Parece que você tem rsync -v. Remova o -ve rsyncficará muito mais silencioso.
Você pode verificar isso observando inicialmente os valores de rsync_short_args e possivelmente rsync_long_args; aqui estão exemplos de minhas configurações:
rsync_short_args -azHS
rsync_long_args --delete --delete-excluded --numeric-ids --fake-super
Se você omitir esses valores, estará confiando que o padrão permanecerá inalterado. O changelog rsnapshotdeve declarar se o padrão foi alterado, mas usando sua própria definição você ignora esse problema completamente.