Skip to content

daemon, seclog: log snapd restarts outside requested restart and standby - #17763

Merged
ernestl merged 2 commits into
canonical:masterfrom
ernestl:ernestl/SNAPDENG-37102/system-restart-error-snapd
Oct 13, 2026
Merged

ernestl merged 2 commits into
canonical:masterfrom
ernestl:ernestl/SNAPDENG-37102/system-restart-error-snapd

Conversation

@ernestl

@ernestl ernestl commented Oct 4, 2026 •

Copy link
Copy Markdown
Member

Log snapd process exits that previously had no lifecycle event. sys_restart_snapd stays reason-only: attributes are snapd_version and reason. There is no error attribute and no error text in the description. The Go error remains in the journal from the process exit.

Rebased on master, which includes the signal event from #17762. A SIGTERM or SIGINT stop is sys_exit_signal_snapd. This change logs sys_restart_snapd for the stop-failed and idle-aborted cases below.

Summary:

  • When idle standby was requested but pending requests prevented it, log reason snapd-idle-aborted. The process exits 0.
  • When Stop returns an error (no Overlord, or tomb.Wait fails outside a system shutdown), log reason snapd-stop-failed.
  • Those two reasons are string constants in seclog. They are not restart.RestartReason values and are not passed to restart.Request.
  • RestartSnapdIdle stays a RestartSocket request. Completed socket standby (ErrRestartSocket, exit 42) still logs sys_standby_snapd with reason snapd-idle.
  • Requested daemon restarts (snapd-update, snapd-revert, snapd-undo, snapd-feature-change) are unchanged. LogSystemRestartSnapd and LogSystemStandbySnapd take the reason as a string.
  • A signal stop stays sys_exit_signal_snapd only. StopDaemon (preseed), system restart and shutdown (doReboot, including a signal that arrives while waiting), and both snap-failure startup results stay off sys_restart_snapd. Those starts are already sys_startup_snapd.
  • The security-logging spread test restores the snapd unit override before starting the service, so SNAPD_STANDBY_WAIT=1s does not remain in an already running process. It still checks sys_exit_signal_snapd, snapd-update, and snapd-idle.

Spec: SD236- Security Event Logging
JIRA: https://warthogs.atlassian.net/browse/SNAPDENG-37102

@github-actions github-actions Bot added Needs Documentation -auto- Label automatically added which indicates the change needs documentation Run only one system Only runs spread tests on one system labels Oct 4, 2026
@ernestl
ernestl force-pushed the ernestl/SNAPDENG-37102/system-restart-error-snapd branch from 0f243c6 to 89c4562 Compare October 4, 2026 22:19
@ernestl
ernestl requested a balanced review from Copilot October 4, 2026 22:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Failure-recovery errors can emit a restart event even though the transient recovery process is not restarted.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
What changed in this PR

Adds security events for unexpected snapd restarts and no-op failure recovery.

Changes:

  • Adds restart reasons and optional error descriptions.
  • Logs restart events before security logging teardown.
  • Extends unit coverage for new restart paths.
File Description
seclog/​seclog.go Supports error-aware restart descriptions.
seclog/​seclog_test.go Tests restart event formatting.
overlord/​restart/​restart.go Defines new restart reasons.
daemon/​daemon.go Logs no-op failure recovery restarts.
daemon/​daemon_test.go Tests failure recovery logging.
cmd/​snapd/​daemon/​main.go Logs unexpected run errors.
cmd/​snapd/​daemon/​main_test.go Tests unexpected restart filtering.
cmd/​snapd/​daemon/​export_test.go Exposes the helper for tests.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread cmd/snapd/daemon/main.go Outdated
@codecov

codecov Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.45455% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 78.50%. Comparing base (ba2436a) to head (a90446e).

Files with missing lines Patch % Lines
daemon/daemon.go 91.66% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #17763      +/-   ##
==========================================
- Coverage   82.06%   78.50%   -3.57%     
==========================================
  Files        1420     1416       -4     
  Lines      201401   201315      -86     
  Branches     2503     2503              
==========================================
- Hits       165279   158039    -7240     
+ Misses      33914    33863      -51     
- Partials     2208     9413    +7205     
Flag Coverage Δ
unittests 78.50% <95.45%> (-3.57%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ernestl
ernestl force-pushed the ernestl/SNAPDENG-37102/system-restart-error-snapd branch from 89c4562 to d278633 Compare October 4, 2026 22:46
@ernestl
ernestl requested a balanced review from Copilot October 4, 2026 22:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Preseed failures currently emit a restart event even though snapd is not restarted.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
Resolved since last review (1)

Comment thread cmd/snapd/daemon/main.go Outdated
@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Tue Oct 13 19:21:05 UTC 2026
The following results are from: https://github.com/canonical/snapd/actions/runs/38382398016

Test Predictor Analysis

Preparing

Test Success %
openstack:ubuntu-core-24-64:tests/main/ 2 times 🟢 100.0%

Executing

Test Success %
openstack:debian-12-64:tests/main/snapshot-basic 🟢 83.3%
openstack:ubuntu-20.04-64:tests/main/lxd 🟢 100.0%

Skipped tests from snapd-testing-skip

If you wish to have any of the below tests run in your PR, in your PR description, add 'unskip:' followed by a copy-and-pasted list of the below tests you wish to run (unskip plus test list must be valid yaml)

  • openstack:arch-linux-64:tests/main/interfaces-location-control
  • openstack:debian-12-64:tests/main/interfaces-location-control
  • openstack:ubuntu-20.04-64:tests/main/interfaces-location-control
  • openstack:ubuntu-24.04-64:tests/main/i18n
  • openstack:ubuntu-24.04-64:tests/main/interfaces-location-control
  • openstack:ubuntu-24.04-64:tests/main/network-retry
  • openstack:ubuntu-26.04-64:tests/main/apparmor-prompting-flag-restart
  • openstack:ubuntu-26.04-64:tests/main/apparmor-prompting-integration-tests
  • openstack:ubuntu-26.04-64:tests/main/apparmor-prompting-prompt-restoration
  • openstack:ubuntu-26.04-64:tests/main/apparmor-prompting-smoke
  • openstack:ubuntu-26.04-64:tests/main/apparmor-prompting-snapd-startup
  • openstack:ubuntu-26.04-64:tests/main/apparmor-prompting-support
  • openstack:ubuntu-26.04-64:tests/main/interfaces-location-control
  • openstack:ubuntu-26.04-64:tests/main/interfaces-requests-activates-handlers

@ernestl ernestl added this to the 2.79 milestone Oct 5, 2026
@ernestl
ernestl force-pushed the ernestl/SNAPDENG-37102/system-restart-error-snapd branch 3 times, most recently from ad5bc34 to c396d5b Compare October 11, 2026 09:25
@ernestl
ernestl requested a balanced review from Copilot October 11, 2026 09:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Restore ordering leaks the temporary standby configuration, and the tomb-failure event path lacks direct coverage.

3 open findings

🧠 Review effort: Balanced

Comment thread tests/main/security-logging/task.yaml Outdated
Comment thread daemon/daemon.go Outdated
@ernestl
ernestl force-pushed the ernestl/SNAPDENG-37102/system-restart-error-snapd branch from c396d5b to 70ce378 Compare October 11, 2026 12:11
@ernestl
ernestl requested a balanced review from Copilot October 11, 2026 12:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The lifecycle paths and their security events are consistently implemented and adequately tested.

1 open finding
2 resolved since last review

🧠 Review effort: Balanced

@ernestl
ernestl force-pushed the ernestl/SNAPDENG-37102/system-restart-error-snapd branch from 70ce378 to 11ec780 Compare October 11, 2026 12:16
@ernestl
ernestl requested a balanced review from Copilot October 11, 2026 12:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The lifecycle paths are consistently implemented and covered by focused unit and integration tests.

1 open finding

🧠 Review effort: Balanced

@ernestl
ernestl force-pushed the ernestl/SNAPDENG-37102/system-restart-error-snapd branch 2 times, most recently from 602874e to 17a6da6 Compare October 11, 2026 18:05
@ernestl
ernestl force-pushed the ernestl/SNAPDENG-37102/system-restart-error-snapd branch 4 times, most recently from c57d073 to 971dcc6 Compare October 11, 2026 19:32
@ernestl ernestl closed this Oct 11, 2026
@ernestl ernestl reopened this Oct 11, 2026
@ernestl
ernestl force-pushed the ernestl/SNAPDENG-37102/system-restart-error-snapd branch 3 times, most recently from a94949f to 5689b6a Compare October 11, 2026 20:29
@ernestl ernestl changed the title many: log error based restarts, and failure recovery restart daemon, seclog: log snapd restarts outside requested restart and standby Oct 11, 2026
@ernestl
ernestl requested a balanced review from Copilot October 11, 2026 20:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The tomb failure path can emit a false restart event during preseed shutdown.

1 open finding
1 resolved since last review

🧠 Review effort: Balanced

Comment thread daemon/daemon.go
@ernestl
ernestl force-pushed the ernestl/SNAPDENG-37102/system-restart-error-snapd branch from 5689b6a to 241c063 Compare October 11, 2026 20:51
@ernestl
ernestl requested a balanced review from Copilot October 11, 2026 20:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The lifecycle logging behavior is consistent with the stated scope and is adequately tested.

0 open findings

1 resolved since last review

🧠 Review effort: Balanced

@ernestl
ernestl marked this pull request as ready for review October 13, 2026 08:32
@github-actions github-actions Bot removed the Run only one system Only runs spread tests on one system label Oct 13, 2026
@ernestl ernestl added the Auto rerun spread Auto reruns spread up to 4 times in non-draft PRs w/ >=1 approval and <20 fails in any fund. system label Oct 13, 2026
@ernestl
ernestl force-pushed the ernestl/SNAPDENG-37102/system-restart-error-snapd branch from 241c063 to a90446e Compare October 13, 2026 10:17

@Mohit-Chachada Mohit-Chachada left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@andrewphelpsj andrewphelpsj left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@bboozzoo bboozzoo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ernestl

ernestl commented Oct 13, 2026

Copy link
Copy Markdown
Member Author

Failure analysis: Not related to changes.

@ernestl
ernestl merged commit d01890a into canonical:master Oct 13, 2026
390 of 412 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Auto rerun spread Auto reruns spread up to 4 times in non-draft PRs w/ >=1 approval and <20 fails in any fund. system Needs Documentation -auto- Label automatically added which indicates the change needs documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants