1.4.3.5. Pause pipeline after more than x reports per minute

A sudden spike in errors might just be a temporary outage of the network, for example. In this case the pipeline is paused after x reports per minute. If the errors persist, the pipeline is paused until user action. Then an email is sent to xyz@company.de).

Some details:

  • The function can be activated for service and local dashboard separately

    (Active or ActiveInLocalDashboard)

  • Maximal number of failed report items per minute which triggers the pause can be configured via config key.

    (MaxErrorsPerMinute)

  • Duration of the pause(s) is set in minutes. Multiple pauses possible (e.g. PauseTime0=1, PauseTime1=2, ...)

  • Key 'UserResumeAsFinalPause' as option => Pipeline will be paused until a user resumes it

  • A pipeline which was resumed manually once, will not be automatically paused again

  • E-mail notification is possible (only in service, not local). E-mail notification only for pipelines which are 'paused until user resume'

    Example:

    Info mail

    Info mail

  • Dashboard shows 'automatically paused' state. Contains also tooltips with more details.

    Message: Job automatically resumed

    Message: Job automatically resumed

    Message: Job not automatically resumed

    Message: Job not automatically resumed

Configuration:

If the e-mail notification is to be activated, the mail server must be configured under $CADENAS_SETUP/reportstask.cfg must be configured.

[ReportsMailServer]
#:HELP;Hostname of mailserver. E.g.: hostname, hostname:port, smtp://<username>
 :<password>@<hostname>:<port>?key1=val&key2=val, ssmtp://<username>
 :<password>@<hostname>:<port>?key1=val&key2=val
Host=
#:HELP;UserName used for authentication to the mailserver (if any)
UserName=
#:HELP;Password used for authentication to the mailserver (if any)
Password=
#:HELP;Set to true if the password is encrypted (use cnstools.helpfu to encrypt password).
 Default = false
PasswordEncrypted=

=> These settings will also be used for the ReportsTask Mail configuration. This way the mail server only needs to be configured once.

Make the specific settings under $CADENAS_SETUP/partwarehouse/pipeline.cfg to make the specific settings.

[ReportWatcher]
#:Help;Activates the ReportWatcher (allows to pause pipelines if there are a lot of errors 
 in a short period of time) (default is false)
Active=
#:Help;Activates the ReportWatcher also in the LocalDashboard environment (default is false)
ActiveInLocalDashboard=
#:Help;Threshold for max errors per minute. If exceeded the pipeline is paused (default is 60)
MaxErrorsPerMinute=
#:Help;List of timeouts (in minutes). First value is used the first time a pipeline is paused, 
 the second one is used the second time the same pipeline is used, etc.
PauseTime0=
#PauseTime1=
#:Help;If true, then the pipeline needs to be resumed manual (After last PauseTime was used). 
 Only in service, not in LocalDashboard (default is false)
UserResumeAsFinalPause=
#:Help;Should an email be send when a pipeline was paused and user resume is required. Also
 see ReportWatcherMail block (default is false). If true then Mailer has to be configured.
SendMail=

Multiple timeouts: A pipeline can be paused multiple times (up to 5):

E.g.: PauseTime0=1 and PauseTime1=2: In this case, the pipeline is paused for one minute the first time and 2 minutes the second time.

UserResumeAsFinalPause: If "true", the pipeline is not automatically resumed, but the user must manually resume the pipeline after the last pause.

Mailer-specific options for the ReportWatcher (also in $CADENAS_SETUP/partwarehouse/pipeline.cfg) ... only needs to be configured if e-mails are to be sent.

[ReportWatcherMailer]
#:HELP;The mail address which should be shown as sender of the mail
Sender=
#:HELP;Comma separated list of mail addresses which should be set as mail receivers
Receivers=
#:HELP;Sender Envelope of the mail
SenderEnvelope=
#:HELP;Define 'ReplyTo' mail address (optional)
ReplyTo=
#:HELP;Comma separated list of mail addresses which should be used as CC (optional)
CcList=
#:HELP;Comma separated list of mail addresses which should be used as BCC (optional)
BccList=
#:HELP;Subject of the outgoing mail
Subject=