Compare commits

..

25 Commits

Author SHA1 Message Date
vorotamoroz
5b19f4415d Refresh issue reporting guidance 2026-08-08 15:10:54 +00:00
vorotamoroz
047429033f Merge pull request #1090 from vrtmrz/1_0_9
Releasing 1.0.9
2026-08-08 22:43:59 +09:00
vorotamoroz
9765569bb6 Add personal note for 1.0.9 2026-08-08 13:04:48 +00:00
github-actions[bot]
c5835a9da6 Releasing 1.0.9 2026-08-08 13:00:23 +00:00
vorotamoroz
8311060f77 Merge pull request #1089 from vrtmrz/reconcile/1.0.8-pre-release
Record the unpromoted 1.0.8 pre-release
2026-08-08 21:47:44 +09:00
vorotamoroz
13d9624407 Record unpromoted 1.0.8 pre-release 2026-08-08 12:33:42 +00:00
vorotamoroz
76560e3bf2 Merge pull request #1083 from calvinbui/fix/qr-aggregator-special-characters
Fix special characters in multi-part settings QR codes
2026-08-08 21:19:22 +09:00
vorotamoroz
1e190d042c Merge main into multi-part settings QR fix 2026-08-08 12:13:31 +00:00
vorotamoroz
40215032dd Merge pull request #1088 from vrtmrz/fix/fast-fetch-page-timeout
Complete bounded Fast Fetch pages on CouchDB 3.2
2026-08-08 20:39:19 +09:00
vorotamoroz
fd9a9175dd Use Commonlib 0.1.8 for reliable Fast Fetch pagination 2026-08-08 11:22:34 +00:00
vorotamoroz
1dfdb72fbd Merge pull request #1085 from vrtmrz/fix/fast-fetch-bounded-pages
Complete bounded Fast Fetch and stop repeated cancelled Fetch
2026-08-08 17:02:05 +09:00
vorotamoroz
23d9fa360d Use Commonlib 0.1.7 for bounded Fast Fetch 2026-08-08 07:44:33 +00:00
vorotamoroz
070b63c952 Document bounded Fast Fetch completion 2026-08-08 05:39:49 +00:00
vorotamoroz
d37af53858 Stop repeating cancelled scheduled Fetch 2026-08-08 04:29:57 +00:00
github-actions[bot]
b05309ad9c Releasing 1.0.7 2026-08-08 02:51:53 +00:00
vorotamoroz
92e01a17e7 Merge pull request #1082 from vrtmrz/fix/fast-fetch-persistence
Keep incomplete Fast Setup isolated until Fast Fetch completes
2026-08-08 11:11:03 +09:00
vorotamoroz
b3030bba53 Document Fast Setup failure recovery 2026-08-08 02:05:42 +00:00
Calvin Bui
cf5181bb28 Fix special characters in aggregated QR settings 2026-08-07 21:48:14 +10:00
vorotamoroz
002cf57116 Use Commonlib 0.1.6 for Fast Fetch 2026-08-07 09:35:25 +00:00
vorotamoroz
15abe344bb Keep incomplete Fast Setup isolated 2026-08-07 08:27:04 +00:00
vorotamoroz
a9e64860d5 Merge pull request #1081 from vrtmrz/1_0_6
Releasing 1.0.6
2026-08-06 21:51:24 +09:00
vorotamoroz
741551be3a Releasing 1.0.6 2026-08-06 10:29:58 +00:00
vorotamoroz
5f69d301f0 Merge pull request #1080 from vrtmrz/fix/setup-synchronisation-settings-guidance
Distinguish absent and unreadable remote settings during setup
2026-08-06 18:57:52 +09:00
vorotamoroz
b42df389fb Distinguish absent and unreadable remote settings during setup
Use typed Commonlib outcomes to guide Fetch and Rebuild without treating a new remote as a failed read. Preserve automatic synchronisation choices when scheduled initialisation is cancelled.
2026-08-06 08:34:41 +00:00
vorotamoroz
6a46966957 Merge pull request #1079 from vrtmrz/1_0_5
Releasing 1.0.5
2026-08-05 20:22:10 +09:00
32 changed files with 1077 additions and 308 deletions

View File

@@ -8,7 +8,7 @@ assignees: ''
---
Thank you for taking the time to report this issue!
Before filling in this form, please read: [How to report an issue](../docs/to_issue_reporting.md).
Before filling in this form, please read [How to report an issue](https://github.com/vrtmrz/obsidian-livesync/blob/main/docs/to_issue_reporting.md).
Issues with sufficient information will be prioritised.
@@ -49,13 +49,14 @@ To get it: open the command palette → "Show debug info".
</details>
### LiveSync version
The hatch report (below) includes version information. If you cannot provide the report, please fill in the version here.
The full LiveSync report below includes version information. If you cannot provide the report, please fill in the version here.
- Self-hosted LiveSync version: <!-- e.g. 0.23.0 — find it in Obsidian Settings → Community Plugins -->
- Self-hosted LiveSync version: <!-- Find it in Obsidian Settings → Community plugins. -->
### Report and Logs from LiveSync
Perform a `Generate full report for opening the issue with debug info` command and provide the generated report. This contains detailed information and recent 1000 log lines, which is very helpful for debugging. **PLEASE AMEND THE REPORT TO REMOVE ANY SENSITIVE INFORMATION BEFORE PASTING.**
If too large to paste here, upload to [Gist](https://gist.github.com/) and share the link.
Run `Generate full report for opening the issue with debug info` and provide the generated report. It contains detailed information and up to 1,000 recent log lines. Review the complete output, and remove credentials, private remote details, Vault names, file paths, file contents, and other private information before sharing it.
If the report is too large to paste here, upload the redacted report to [Gist](https://gist.github.com/) and share the link.
<details>
<summary>Report and Logs (primary)</summary>

View File

@@ -33,7 +33,8 @@
const id = params.get('id');
const total = parseInt(params.get('n') || '0');
const index = parseInt(params.get('i') || '-1');
const data = params.get('d');
// Keep the chunk percent-encoded so URI delimiters remain part of the settings payload.
const data = hash.match(/(?:^|&)d=([^&]*)/)?.[1];
const app = document.getElementById('app');

View File

@@ -0,0 +1,351 @@
# Architectural Decision Record: Fast Fetch Persistence and Completion Semantics
## Status
Accepted
## Context
Fast Fetch accelerates Fast Setup (Simple Fetch) by reading CouchDB's continuous
changes feed directly, decrypting each document, and writing batches to the local
database. It then allows LiveSync to reflect the completed database into the
Vault.
This path deliberately bypasses PouchDB's ordinary replication machinery. It
must therefore reproduce the correctness guarantees on which the rest of the
initialisation workflow relies:
- a remote document is decrypted and validated before it is written locally;
- every result from a batch write is checked;
- a checkpoint represents the last contiguous remote sequence which is durable
in the local database; and
- successful completion requires CouchDB to terminate every finite changes page,
every returned row to be durable, and a subsequent normal probe to report no
available rows.
The existing implementation combines line parsing, decryption, persistence, and
completion checks within one broad error handler. A decryption or persistence
failure can consequently be reported as a malformed JSON line and skipped. A
batch write can also resolve while containing individual failed results. In both
cases the stream may continue, advance its checkpoint incorrectly, or wait
indefinitely for a completion condition which the failed row would have
satisfied.
Fast Fetch also reads the normal changes feed before opening each continuous
page. A normal response's `pending` value counts items which remain after the
response's `results`, so `pending` alone is not the available workload. With a
one-row probe, the page can contain `results.length + pending` rows.
The documented `limit=0` behaviour cannot be used as a portable zero-payload
probe. CouchDB's API documentation says that `limit=0` has the same effect as
`limit=1`, while CouchDB 3.5.0 with a two-shard database was observed to return
no result rows and leave the complete count in `pending`. Fast Fetch therefore
uses an explicit one-row normal probe and includes no document bodies.
Finite continuous-feed completion differs across supported CouchDB releases.
CouchDB 3.5.0 was observed to close a heartbeat-enabled feed with a
`{ "last_seq": ... }` line when its finite `limit` is met. CouchDB 3.2 instead
continues to wait for database updates after the limit has been consumed. With
a heartbeat configured, each wait emits another heartbeat and the page can
remain open indefinitely, even after every requested row has arrived.
On CouchDB 3.2, an explicit `timeout` without a heartbeat has different
semantics from a total request deadline. Shard-result waits may emit blank
keep-alive lines and continue processing. Once the currently available changes
have been exhausted and the feed is waiting for another database update, the
timeout stops that wait and returns the feed-level `last_seq`. The timeout can
therefore terminate a finite page without limiting the duration of an active
page transfer.
The CouchDB sequence token is opaque and must be handled using CouchDB's
sequence semantics, without parsing, ordering, or comparison. On clustered
CouchDB, a changes row and the feed-level `last_seq` may encode related
positions with different opaque tokens. Separate requests are also not one
locked snapshot: their rows may be partially ordered, and replica failover may
repeat changes. Fast Fetch must therefore be idempotent and must use each
terminal `last_seq` only by returning it to CouchDB as the next `since` value.
## Decision
### Remote page sizing and completion
Fast Fetch obtains an approximate progress target from a normal changes-feed
request with `since=now`, `limit=1`, and `include_docs=false`. This token is for
progress reporting only. It is not compared with any other token and is not
used as a completion checkpoint.
Before every bounded page, Fast Fetch requests a normal changes feed from the
current durable cursor with `limit=1` and `include_docs=false`. The probe and
the following continuous page use the same `since`, style, and filter
selection. Reading the probe does not consume rows from CouchDB; the continuous
request starts again from that same cursor.
The number currently available is `results.length + pending`. If it is zero,
Fast Fetch is caught up and completes without opening another stream. Otherwise,
the next continuous request uses the smaller of that count and 10,000 as its
finite `limit`.
Each finite page omits `heartbeat` and sets `timeout=1000`. This lets CouchDB
3.2 return the page's terminator one second after it exhausts the currently
available changes, rather than keeping the request open for future writes. The
client immediately reconnects from that terminator while another normal probe
reports available work. This bounded cycle also preserves the intent of the
earlier iOS and iPadOS heartbeat workaround: Fast Fetch no longer depends on a
silent continuous request eventually closing at CouchDB's default 60-second
timeout.
The probe and page are separate HTTP requests, not a transactional snapshot.
New writes, replica selection, or administrative changes may alter the rows
between them. A page which returns at least one row and a valid terminator may
therefore be shorter than the probe's estimate. Fast Fetch persists that page
and probes again. A page which terminates without making progress after a
positive probe is a retryable transport failure, avoiding an unbounded busy
loop.
Each continuous request ends with its own `{ "last_seq": ... }` line. Fast
Fetch treats that line separately from a changes row, flushes and validates all
preceding local writes, and only then persists the opaque `last_seq`. The exact
value is replayed as the next request's `since`; it is never parsed, ordered, or
compared with a row's `seq`, another request's `last_seq`, or the database's
`update_seq`.
The limit counts outer changes-result rows. A tombstone is one row and consumes
one page slot even when no document body is present. With `style=all_docs`,
multiple leaf revisions inside one row's `changes` array do not consume
additional slots. Changing `include_docs` between the lightweight probe and the
document-bearing continuous page changes the payload, not the row selection.
### Processing and persistence
Each non-blank line from the changes feed is processed through these ordered
stages:
1. parse and validate the changes-feed row;
2. decrypt and validate its document, when a document is present;
3. add the document to the pending local batch;
4. persist the batch;
5. inspect every result returned by the batch write; and
6. after the finite page ends, persist its `last_seq` terminator.
With `new_edits: false`, PouchDB follows CouchDB behaviour and may omit successful
results. Fast Fetch therefore inspects every returned result and treats any
error result as a failed batch. An empty result is valid when PouchDB accepted
the complete batch.
The checkpoint may advance only to the last contiguous sequence for which all
preceding documents are durable. A row which legitimately requires no local
write may advance the checkpoint only after any preceding buffered documents
have been flushed successfully. A page's `last_seq` is committed under the same
rule before that page reports success.
If a batch is partly written, its checkpoint is not advanced. Retrying the batch
with `new_edits: false` is expected to be idempotent, including for documents
which were accepted during the first attempt.
Blank heartbeat lines are ignored. Malformed rows are failures; they are not
silently skipped. Logs may describe the stage and sequence involved, but must
not include the raw changes-feed line because it may be large or sensitive.
Each finite continuous changes request and its decoded reader must be terminated
on every exit. Releasing a reader lock alone does not cancel the underlying
request. Failure and completion paths therefore abort the request and attempt
to cancel the reader before the bounded remote-activity scope ends.
### Failure classification and retry
The streaming boundary returns a small structured failure with a stage and an
explicit retryability decision. The initial stages are:
```typescript
type StreamingFetchFailureStage = "transport" | "authentication" | "protocol" | "decryption" | "storage";
```
This type is an internal behavioural contract, not a user-interface status
enumeration. It may carry safe diagnostic context, such as an HTTP status or a
sequence token, without carrying document content.
Only explicitly recognised transient transport failures are retried
automatically. Examples include an interrupted connection, HTTP 408, HTTP 429,
and selected HTTP 5xx responses. Authentication, protocol, decryption, and local
storage failures are terminal by default. A future implementation may recognise
a narrower retryable case, but it must do so explicitly rather than retry every
exception.
Each retry resumes from the last durable contiguous checkpoint. Retry exhaustion
returns an actionable classified failure to the caller.
### Initialisation lifecycle
Fast Fetch success is the only path which may continue with the offline scan,
finish the rebuild, resume Vault reflection, clear the Fast Fetch checkpoint,
remove the flag file, or forget the remembered initialisation choice.
The LiveSync initialisation boundary uses an explicit suspension policy.
Ordinary Fetch and Rebuild resume Vault reflection when they finish, SCRAM keeps
file watching suspended, and Fast Fetch keeps both file watching and replication
result parsing suspended only when initialisation fails. Fast Fetch asserts both
suspensions before it begins and owns their final state: success clears both,
whereas a false result or exception sets both. This final assignment also covers
a late failure after rebuild finalisation and the legacy
`doNotSuspendOnFetching` path.
On failure:
- the local checkpoint and any durably fetched documents are retained for a
later retry;
- the local database is not marked as resolved;
- Vault reflection remains suspended;
- the offline scan and rebuild finalisation are not run; and
- the flag file and remembered initialisation choice remain available so that
restart recovery can offer the same operation again.
Any bounded remote-activity scope must still be released in a `finally` path, as
defined by [Bounded Remote Activity](2026_07_bounded_remote_activity.md). Keeping
Vault reflection suspended does not permit a wake lock or similar resource to
leak.
## Ownership
The responsibilities are divided at three injectable boundaries.
### Streaming Fetch
The Commonlib streaming implementation owns HTTP response validation, NDJSON
parsing, invocation of the decryption delegate, batch-write result validation,
contiguous checkpoint advancement, finite-page completion, and classified
failures. It does not know about the Vault, setup dialogues, flag files, or
LiveSync settings.
### Rebuilder
The Commonlib rebuilder owns the local database lifecycle, checkpoint storage,
retry policy, marking a completed database as resolved, optional resumption of
reflection, and final checkpoint removal. It does not parse changes-feed rows or
interpret user-interface choices.
### LiveSync Fast Setup
LiveSync owns the setup choices, suspension of initial Vault reflection,
invocation of Fast Fetch, the success-only offline scan and rebuild finalisation,
and cleanup of flag files and remembered choices. It does not reinterpret
document, encryption, or storage failures as successful setup.
## Non-Goals
This decision does not:
- change the Metadata and Chunks formats, encryption scheme, security seed, or
path obfuscation;
- change ordinary PouchDB replication, Standard Fetch, or offline-scan
semantics;
- provide a transaction spanning CouchDB and the local database;
- skip corrupt or unreadable documents and continue with a partial database;
- add an automatic fallback from Fast Fetch to Standard Fetch;
- define the detailed failure dialogue or other setup user-interface changes;
or
- provide a transaction or locked snapshot across the normal probe and the
following continuous page.
An explicit Standard Fetch choice remains available when a user needs the
ordinary replication path. Any automatic fallback or richer recovery dialogue
requires a separate decision because it changes user-visible setup behaviour.
## Verification
The implementation is verified primarily with London School interaction tests,
using mocks at each owned boundary to prove collaboration and call order.
### Streaming Fetch unit tests
Inject the HTTP stream, decryption delegate, local batch writer, and checkpoint
writer. Verify that:
- the order is decrypt, persist, inspect results, then checkpoint;
- parsing, decryption, and batch-result failures prevent checkpoint advancement
and completion;
- a partly failed batch leaves the checkpoint unchanged and reports a storage
failure;
- rows without a local write flush earlier buffered documents before advancing;
- a returned probe row is counted in addition to `pending`, including when
`pending` is zero;
- every probe uses `limit=1`, excludes document bodies, and is repeated from the
previous page's opaque terminator;
- each bounded page omits `heartbeat`, uses `timeout=1000`, and can complete
under CouchDB 3.2 after its current rows have been delivered;
- deletion and document-less rows consume a page slot;
- a row count cannot complete a page without its `last_seq` terminator;
- a page terminator cannot advance the checkpoint before its batch is durable;
- a final row and `last_seq` with different opaque representations complete
normally without a token comparison;
- a shorter valid page is persisted and followed by another probe, while a
zero-row page after a positive probe fails without looping;
- workloads over 10,000 rows resume from each durable finite-page checkpoint;
- authentication and malformed-protocol responses are terminal;
- recognised transient transport failures are classified as retryable; and
- diagnostics do not log the raw changes-feed line.
### Rebuilder unit tests
Inject the streaming operation and lifecycle collaborators. Verify that:
- transient failures retry from the latest durable checkpoint;
- terminal failures are attempted once;
- success marks the database as resolved, resumes reflection when requested, and
clears the checkpoint in that order; and
- failure does not mark the database as resolved, resume reflection, or clear
the checkpoint.
### LiveSync orchestration unit tests
Inject Fast Fetch, the offline scanner, rebuild finalisation, and cleanup
collaborators. Verify that failure performs none of the success-only actions and
leaves initial Vault reflection suspended. Verify that success retains the
existing setup sequence and cleanup.
### Integration and E2E tests
Commonlib's CouchDB integration test remains responsible for the real HTTP
changes feed, opaque sequence tokens, deletion rows, and local batch
persistence. It should use the maintained CI CouchDB release, a two-shard
database, and a data set large enough to cross a local batch boundary, and
confirm that the final checkpoint can be passed back to CouchDB as `since` with
no result rows or pending changes. The test must not compare that token's
representation with a separately requested target or changes-row token. The
focused regression test covers CouchDB 3.2's page-tail behaviour; compatibility
with a real CouchDB 3.2 server can be confirmed manually without expanding the
permanent CI matrix.
LiveSync's real Obsidian Setup URI workflow remains responsible for the actual
Fast Fetch selection, E2EE passphrase, Vault reflection, ordinary file round
trip, and hidden-file synchronisation. Injected parsing, decryption, and
persistence failures remain unit-test responsibilities; repeating them through
the real Obsidian E2E does not add coverage for an unchanged framework boundary.
This follows [Real Obsidian E2E](2026_06_real_obsidian_e2e.md).
## Consequences
- A deterministic document failure which previously appeared to be skipped now
fails Fast Fetch. This is an intentional safety improvement because the local
database is known to be incomplete.
- Partial durable work and its contiguous checkpoint can be reused by a later
attempt without exposing the partial database to the Vault.
- Retry delays are no longer spent on authentication, corrupt content, protocol,
or local persistence failures which cannot repair themselves.
- Progress totals remain approximate and may grow when a later probe observes
new work, without affecting correctness.
- A completed page can spend up to one second waiting for its terminator before
Fast Fetch probes and reconnects. Active page transfer is not constrained to
one second.
- The implementation requires coordinated changes in Commonlib and LiveSync.
Commonlib remains the authoritative package for streaming and rebuilder
behaviour; LiveSync consumes an immutable Commonlib release and owns its setup
orchestration.
- Ordinary replication remains unchanged and continues to provide the reference
correctness contract for decrypting, persisting, and checkpointing replicated
documents.
## References
- [Apache CouchDB changes-feed API](https://docs.couchdb.org/en/stable/api/database/changes.html)
- [Apache CouchDB 2.0 upgrade notes for opaque update sequences](https://docs.couchdb.org/en/stable/whatsnew/2.0.html#upgrade-notes)
- [Apache CouchDB replication protocol](https://docs.couchdb.org/en/stable/replication/protocol.html)

View File

@@ -39,7 +39,7 @@ Use this path only when the remote database is new, or when this device is inten
![Final server overwrite warning](../images/quick-setup/guide-quick-setup-first-rebuild-confirmation.png)
8. A newly provisioned database may show `Fetch Remote Configuration Failed` because it does not contain a saved preferred configuration yet. If this is a genuinely new setup, select `Skip and proceed`. Otherwise, stop and investigate before continuing.
8. A newly provisioned database may show `No Synchronisation Settings Found`. Select `Use this device's settings` for the intended new database. If you expected existing settings, cancel and check the selected remote. Cancellation restarts with Vault and database reflection paused; keep LiveSync disabled until the remote has been checked. If `Could Not Read Synchronisation Settings` appears, retry, or use this device's settings to continue the overwrite. The overwrite still requires a working connection.
![Expected missing remote configuration choice for a new database](../images/quick-setup/guide-quick-setup-missing-remote-configuration.png)
@@ -134,7 +134,7 @@ Use this path when CouchDB is ready but a Setup URI is unavailable. It configure
8. Select `Create or connect to database and continue`. Onboarding requires this connection test to succeed.
9. Review `Setup Complete: Preparing to Initialise Server`, then select `Restart and Initialise Server`.
10. Read the final overwrite warning. Select `I Understand, Overwrite Server` only when this device is intentionally the source of truth and a current backup exists.
11. A newly created database can show `Fetch Remote Configuration Failed` because it does not yet contain a saved preferred configuration. Select `Skip and proceed` only for this known new database.
11. A newly created database can show `No Synchronisation Settings Found`. Select `Use this device's settings` for this known new database. If you expected existing settings, cancel and check the selected remote. Cancellation restarts with Vault and database reflection paused; keep LiveSync disabled until the remote has been checked. If `Could Not Read Synchronisation Settings` appears, retry, or use this device's settings to continue the overwrite. The overwrite still requires a working connection.
12. Acknowledge `All optional features are disabled`, then keep Obsidian open until the initialisation progress has cleared.
Create and synchronise an ordinary test note. Once it has reached CouchDB, follow [Create a Setup URI for another device](#create-a-setup-uri-for-another-device), then [Add another device](#add-another-device). This keeps the second device aligned with the remote profile and encryption settings which the first device actually applied.

View File

@@ -68,6 +68,8 @@ On the next start, LiveSync:
4. discards and reconstructs the local LiveSync database from the selected remote; and
5. resumes only after the scheduled operation has completed or been cancelled safely.
Fast Setup retains its fetch flag, last successfully stored remote position, and selected data-processing method when an error while decrypting data, reading the remote response, or writing to the local database stops the reconstruction. File watching and database reflection remain suspended. Review the first specific error in **Show log**, correct its cause, then restart Obsidian to retry from the retained state. Do not remove the fetch flag or resume the Scram switches while you intend to continue the operation. If the same error remains, leave LiveSync suspended and [collect a report](troubleshooting.md#collect-a-report).
For P2P, a source peer must be online, discovered, and selected in `P2P Rebuild`. Merely opening an empty signalling room does not complete Fetch. Closing the rebuild dialogue without selecting a peer reports failure and does not treat the local database as restored.
Review the [Fast Setup guide](tips/fast-setup.md) before using this operation on a Vault which contains unsynchronised local work.
@@ -105,7 +107,7 @@ Create only the flag required for the chosen operation.
| `flag_fetch.md` or `redflag3.md` | Schedule **Reset Synchronisation on This Device** from the selected remote. |
| `flag_rebuild.md` or `redflag2.md` | Schedule **Overwrite Server Data with This Device's Files**, or local P2P preparation when no central remote exists. |
Flag files themselves are excluded from synchronisation. Fetch and rebuild flags are removed by the scheduled workflow after completion or cancellation; `redflag.md` is a manual emergency stop.
Flag files themselves are excluded from synchronisation. Fetch and rebuild flags are removed by the scheduled workflow after completion or safe cancellation. A failed Fast Setup retains its fetch flag so that a later start can retry it; `redflag.md` is a manual emergency stop.
## When the warning continues

View File

@@ -50,7 +50,7 @@ Use a new bucket prefix, or a prefix whose contents you deliberately intend to r
![Final Object Storage overwrite confirmation](../images/object-storage-setup/guide-object-storage-setup-first-rebuild-confirmation.png)
6. A new prefix may show `Fetch Remote Configuration Failed` because it has no saved configuration. Select `Skip and proceed` only for a genuinely new prefix. Otherwise, stop and check the endpoint, credentials, bucket, and prefix.
6. A new prefix may show `No Synchronisation Settings Found`. Select `Use this device's settings` for a genuinely new prefix. If you expected existing settings, cancel and check the endpoint, credentials, bucket, and prefix. Cancellation restarts with Vault and database reflection paused; keep LiveSync disabled until the remote has been checked. If `Could Not Read Synchronisation Settings` appears, retry, or use this device's settings to continue the overwrite. The overwrite still requires a working connection.
![Expected missing remote configuration for a new Object Storage prefix](../images/object-storage-setup/guide-object-storage-setup-missing-remote-configuration.png)

View File

@@ -63,3 +63,11 @@ Once you confirm your choices:
1. The plug-in performs a fast download of the remote database (`fetchLocalDBFast`).
2. It automatically runs a full scan (`synchroniseAllFilesBetweenDBandStorage`) in the foreground to reflect database changes in your local vault files immediately.
3. The plug-in finalises the process and resumes normal operational status.
### If Fast Setup Stops
Fast Setup records the last successfully stored remote position as it saves documents. A transient connection interruption is retried automatically from that position. The operation reports completion only after the captured remote state has been stored successfully.
If an error while decrypting data, reading the remote response, or writing to the local database stops the operation, LiveSync does not run the Vault scan or finalise the reconstructed local database. It retains the saved position, the selected data-processing method, and the fetch flag. File watching and database reflection also remain suspended so that a partly reconstructed database cannot be applied to the Vault or combined with new local changes.
Review the first specific error in **Show log**, correct its cause, then restart Obsidian to retry from the retained state. While you intend to continue the operation, do not remove the fetch flag or manually resume the Scram switches. If the same error remains after a restart, leave LiveSync suspended, preserve the available data, and [collect a report](../troubleshooting.md#collect-a-report).

View File

@@ -1,145 +1,65 @@
# How to report an issue
Thank you for helping improve Self-hosted LiveSync!
Thank you for helping improve Self-hosted LiveSync. A concise report with the right evidence is more useful than trying several recovery operations before reporting the original symptom.
This document explains how to collect the information needed for an issue report. Issues with sufficient information will be prioritised.
Use the [issue report template](https://github.com/vrtmrz/obsidian-livesync/issues/new?template=issue-report.md) for the report itself. Use [Troubleshooting](troubleshooting.md) to diagnose a symptom or choose a recovery action.
---
## Preserve the original symptom
## Filled example
Do not reset a database, rebuild a remote, change transport, or enable P2P merely to see whether the problem disappears. These actions can change the evidence and may make the original cause harder to identify.
Here is an example of a well-filled report for reference.
If the problem may involve data loss, corruption, or unexpected deletion, preserve a copy of every readable affected file and stop editing it on other devices before changing settings.
### Abstract
Include when the problem began, whether it followed an update or restart, how often it occurs, and which device and remote type were involved.
The synchronisation hung up immediately after connecting.
## Required information
### Expected behaviour
### Describe the behaviour
- Synchronisation ends with the message `Replication completed`
- Everything synchronised
Complete the issue template with:
### Actually happened
- a one- or two-sentence summary;
- the expected and actual behaviour;
- repeatable steps, or the frequency and timing when reliable reproduction is not available; and
- the role of each relevant device, such as the device where the change originated and the device where the failure appeared.
- Synchronisation was cancelled with the message `TypeError: Failed to fetch` (visible in the plug-in log around lines 1012)
- No files synchronised
### Obsidian debug information
### Reproducing procedure
Open the command palette with `Ctrl`+`P` or `Command`+`P`, run `Show debug info`, and include its output for each relevant device. The device where the problem appeared is required. Information from the other participating devices is particularly useful for synchronisation problems.
1. Configure LiveSync with the settings shown in the attached report.
2. Click the sync button on the ribbon.
3. Synchronisation begins.
4. About two or three seconds later, the error `TypeError: Failed to fetch` appears.
5. Replication stops. No files synchronised.
### Full LiveSync report
### Obsidian debug info (Device 1 — Windows desktop)
Run `Generate full report for opening the issue with debug info` on the device where the problem appeared. For a synchronisation problem, also collect a report from another participating device when its settings or logs are relevant. The command copies the current LiveSync settings summary and up to 1,000 recent log lines. It collects verbose log lines even when `Verbose Log` is disabled, so you do not need to enable that setting before reproducing the problem.
```
SYSTEM INFO:
Obsidian version: v1.2.8
Installer version: v1.1.15
Operating system: Windows 10 Pro 10.0.19044
Login status: logged in
Catalyst license: supporter
Insider build toggle: off
Community theme: Minimal v6.1.11
Snippets enabled: 3
Restricted mode: off
Plugins installed: 35
Plugins enabled: 11
1: Self-hosted LiveSync v0.19.4
...
```
The command automatically redacts known credential fields in the settings summary. It cannot guarantee that private text in log messages or unrecognised configuration fields is removed. Review the complete output before sharing it. Remove or replace:
### Report from LiveSync
- usernames, passwords, passphrases, tokens, keys, and custom headers;
- private server URLs, network addresses, database names, bucket names, room identifiers, and relay details;
- Vault names, device names, and file paths; and
- file contents or other private text which appears in a log message.
```
----remote config----
cors:
credentials: "true"
...
---- Plug-in config ---
couchDB_URI: self-hosted
couchDB_USER: 𝑅𝐸𝐷𝐴𝐶𝑇𝐸𝐷
...
```
Document and chunk identifiers can also be private metadata, but they may be necessary for diagnosing file reconstruction and chunk availability. Decide deliberately whether to share them. If you remove them, state that the report was redacted and that this may limit the diagnosis.
### Plug-in log
For a large report, you may share a GitHub Gist after reviewing and redacting it. Deleting a Gist later cannot undo information which has already been disclosed.
```
2023/5/24 10:50:33->HTTP:GET to:/ -> failed
2023/5/24 10:50:33->TypeError:Failed to fetch
2023/5/24 10:50:33->could not connect to https://example.com/ : your vault
(TypeError:Failed to fetch)
```
## Additional evidence when relevant
---
### A problem involving one file
## How to collect each piece of information
Run `Copy database information for the active file`, or use **Hatch****Copy database information for a file** to select another file.
### Obsidian debug info
This report describes only the local database on that device. It includes the Vault-relative path, document and chunk identifiers, local revisions, conflicts, and local chunk availability. It does not query the remote or include file contents. Review paths and identifiers as private metadata before sharing them.
Open the command palette (`Ctrl/Cmd + P`) and run **"Show debug info"**. Copy the output and paste it into the issue.
### A problem which crosses a restart
If multiple devices are involved in the problem (e.g., sync between a phone and a desktop), please provide the debug info for each device. The device where the issue occurred is required; information from other devices is strongly recommended.
Use `Write logs into the file` under **Hatch** only when the in-memory report cannot cover the restart. Persistent logging affects performance and can record private information. Disable it after reproducing the problem, review the log before sharing it, and remove the log file when it is no longer needed.
### Report from LiveSync (hatch report)
### A connection, authentication, or CORS problem
1. Open LiveSync settings.
2. Go to the **Hatch** pane.
3. Press the **Make report** button.
Include network evidence only when the ordinary LiveSync log cannot show the rejected response. Follow [Inspect a network failure](troubleshooting.md#inspect-a-network-failure), and remove request paths, remote addresses, authority and authorisation values, cookies, credentials, payload identifiers, and response secrets before sharing screenshots or copied data.
The report will be copied to your clipboard. It contains your LiveSync configuration and the remote server configuration, with credentials automatically redacted.
## Sharing the report
**Tip:** For large reports, consider uploading to [GitHub Gist](https://gist.github.com/) and sharing the link instead of pasting directly into the issue. This makes it easier to manage, and if you accidentally leave sensitive data in, a Gist can be deleted.
Paste reports into the matching collapsible sections in the issue template, or provide a link to an already-redacted Gist. A separate plug-in log is normally unnecessary because the full LiveSync report already contains the recent verbose log history.
If you paste directly, wrap it in a `<details>` tag to keep the issue readable:
```
<details>
<summary>Report from hatch</summary>
```
----remote config----
:
```
</details>
```
### Plug-in log
The plug-in log is volatile by default (not saved to disk) and shown only in the log dialogue, which can be opened by tapping the **document box icon** in the ribbon.
#### Enable verbose log
Before reproducing the issue, enable **Verbose Log** in LiveSync's **General Settings** pane. Without this, many diagnostic messages will be suppressed.
#### Persist the log to a file (optional)
If you need to capture a log across a restart, enable **"Write logs into the file"** in General Settings. Note that log files may contain sensitive information — use this option only for troubleshooting, and disable it afterwards.
As with the hatch report, consider uploading large logs to [GitHub Gist](https://gist.github.com/).
### Network log (for connection-related issues only)
If the issue is related to network connectivity (e.g., cannot connect to the server, authentication errors), a network log captured from browser DevTools can be very helpful. You do not need to include this for non-connection issues.
#### Opening DevTools
| Platform | Shortcut |
|----------|----------|
| Windows / Linux | `Ctrl + Shift + I` |
| macOS | `Cmd + Shift + I` |
| Android | Use [Chrome remote debugging](https://developer.chrome.com/docs/devtools/remote-debugging/) |
| iOS | Use [Safari Web Inspector](https://developer.apple.com/documentation/safari-developer-tools/inspecting-ios) on a Mac |
#### What to capture
1. Open the **Network** pane in DevTools.
2. Reproduce the issue.
3. Look for requests marked in red.
4. Capture screenshots of the **Headers**, **Payload**, and **Response** tabs for those requests.
**Important — redact before sharing:**
- Headers: conceal the request URL path, Remote Address, `authority`, and `authorisation` values.
- Payload / Response: the `_id` field contains your file paths — redact if needed.
If a maintainer asks for a more specialised diagnostic, collect only that additional evidence and review it again before publishing it.

View File

@@ -117,6 +117,8 @@ Enable Obsidian's `Detect all file extensions`, then check LiveSync selectors, i
## Collect a report
Follow [How to report an issue](to_issue_reporting.md) for the complete reporting checklist, including Obsidian debug information and the privacy review required before sharing evidence.
Run `Generate full report for opening the issue with debug info` to copy the current settings summary and recent verbose log lines. Remove credentials, remote URLs, Vault names, file contents, and other private information before sharing it.
When a problem concerns one file, run **Copy database information for the active file**, or use **Hatch****Copy database information for a file** to select another file. The report describes this device's local database view, including the Vault-relative path, document and chunk identifiers, local database revisions, conflicts, and local chunk availability. It does not query the remote server or include file contents. Treat paths and identifiers as private metadata before sharing.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 19 KiB

View File

@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "1.0.5",
"version": "1.0.9",
"minAppVersion": "1.7.2",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",

18
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{
"name": "obsidian-livesync",
"version": "1.0.5",
"version": "1.0.9",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "obsidian-livesync",
"version": "1.0.5",
"version": "1.0.9",
"license": "MIT",
"workspaces": [
"src/apps/cli",
@@ -23,7 +23,7 @@
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.4",
"@vrtmrz/livesync-commonlib": "0.1.8",
"@vrtmrz/obsidian-plugin-kit": "0.1.3",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",
@@ -4775,9 +4775,9 @@
}
},
"node_modules/@vrtmrz/livesync-commonlib": {
"version": "0.1.4",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.4.tgz",
"integrity": "sha512-qQst1QCZEgfxwpdjnjFmXE3JNy4ATKhN8ihJSUUho4mdWC1O4zyRRhmQjg0GHu5hQPZokufr2mrP4Hur8niDNg==",
"version": "0.1.8",
"resolved": "https://registry.npmjs.org/@vrtmrz/livesync-commonlib/-/livesync-commonlib-0.1.8.tgz",
"integrity": "sha512-Kn1AF41h2Dog37ThU7KgLcKxItCCerLEBWg1eSGAUoTk3TyPBYynvtmVFwWhy6LCePcuwB/+x7EQNTL3Sgzkyg==",
"license": "MIT",
"dependencies": {
"@aws-sdk/client-s3": "^3.808.0",
@@ -15924,7 +15924,7 @@
},
"src/apps/cli": {
"name": "self-hosted-livesync-cli",
"version": "1.0.5-cli",
"version": "1.0.9-cli",
"dependencies": {
"chokidar": "^4.0.0",
"minimatch": "^10.2.5",
@@ -15949,7 +15949,7 @@
},
"src/apps/webapp": {
"name": "livesync-webapp",
"version": "1.0.5-webapp",
"version": "1.0.9-webapp",
"dependencies": {
"octagonal-wheels": "^0.1.52"
},
@@ -15961,7 +15961,7 @@
}
},
"src/apps/webpeer": {
"version": "1.0.5-webpeer",
"version": "1.0.9-webpeer",
"dependencies": {
"octagonal-wheels": "^0.1.52"
},

View File

@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "1.0.5",
"version": "1.0.9",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",
@@ -177,7 +177,7 @@
"@smithy/types": "^4.14.3",
"@smithy/util-retry": "^4.4.5",
"@vrtmrz/browser-ui-kit": "0.1.0",
"@vrtmrz/livesync-commonlib": "0.1.4",
"@vrtmrz/livesync-commonlib": "0.1.8",
"@vrtmrz/obsidian-plugin-kit": "0.1.3",
"@vrtmrz/ui-interactions": "0.1.2",
"diff-match-patch": "^1.0.5",

View File

@@ -1,7 +1,7 @@
{
"name": "self-hosted-livesync-cli",
"private": true,
"version": "1.0.5-cli",
"version": "1.0.9-cli",
"main": "dist/index.cjs",
"type": "module",
"scripts": {

View File

@@ -1,7 +1,7 @@
{
"name": "livesync-webapp",
"private": true,
"version": "1.0.5-webapp",
"version": "1.0.9-webapp",
"type": "module",
"description": "Browser-based Self-hosted LiveSync using FileSystem API",
"scripts": {

View File

@@ -1,7 +1,7 @@
{
"name": "webpeer",
"private": true,
"version": "1.0.5-webpeer",
"version": "1.0.9-webpeer",
"type": "module",
"scripts": {
"dev": "vite",

View File

@@ -49,6 +49,18 @@ export const liveSyncProvisionalEnglishMessages = {
"Connect to existing database and continue": "Connect to existing database and continue",
"Test connection and save": "Test connection and save",
"Save without connecting": "Save without connecting",
"Use this device's settings": "Use this device's settings",
Retry: "Retry",
"No Synchronisation Settings Found": "No Synchronisation Settings Found",
"The selected remote has no saved synchronisation settings. This is normal for a new remote. Use this device's settings, or cancel if you expected existing settings.":
"The selected remote has no saved synchronisation settings. This is normal for a new remote. Use this device's settings, or cancel if you expected existing settings.",
"Could Not Read Synchronisation Settings": "Could Not Read Synchronisation Settings",
"Could not read the remote's synchronisation settings. Check the connection and credentials, then retry.":
"Could not read the remote's synchronisation settings. Check the connection and credentials, then retry.",
"Could not read the remote's synchronisation settings. Retry, or continue the overwrite with this device's settings. A working connection is still required.":
"Could not read the remote's synchronisation settings. Retry, or continue the overwrite with this device's settings. A working connection is still required.",
"Skips checking and applying synchronisation settings from the remote.":
"Skips checking and applying synchronisation settings from the remote.",
"Enter a complete HTTP or HTTPS URL.": "Enter a complete HTTP or HTTPS URL.",
"CouchDB validates the database name when you connect. The name must not be empty.":
"CouchDB validates the database name when you connect. The name must not be empty.",

View File

@@ -10,6 +10,8 @@ import {
type RemoteDBSettings,
IncompatibleChangesInSpecificPattern,
CompatibleButLossyChanges,
type RemotePreferredTweakResult,
RemotePreferredTweakStatuses,
} from "@vrtmrz/livesync-commonlib/compat/common/types";
import { escapeMarkdownValue } from "@vrtmrz/livesync-commonlib/compat/common/utils";
import { AbstractModule } from "@/modules/AbstractModule.ts";
@@ -256,22 +258,21 @@ export class ModuleResolvingMismatchedTweaks extends AbstractModule {
return "IGNORE";
}
async _fetchRemotePreferredTweakValues(trialSetting: RemoteDBSettings): Promise<TweakValues | false> {
const replicator = await this.services.replicator.getNewReplicator(trialSetting);
if (!replicator) {
this._log("The remote type is not supported for fetching preferred tweak values.", LOG_LEVEL_NOTICE);
return false;
}
if (await replicator.tryConnectRemote(trialSetting)) {
const preferred = await replicator.getRemotePreferredTweakValues(trialSetting);
if (preferred) {
return preferred;
async _fetchRemotePreferredTweakValues(trialSetting: RemoteDBSettings): Promise<RemotePreferredTweakResult> {
try {
const replicator = await this.services.replicator.getNewReplicator(trialSetting);
if (!replicator) {
this._log("The remote type does not support preferred tweak values.", LOG_LEVEL_NOTICE);
return { status: RemotePreferredTweakStatuses.UNSUPPORTED };
}
this._log("Failed to get the preferred tweak values from the remote server.", LOG_LEVEL_NOTICE);
return false;
return await replicator.getRemotePreferredTweakValues(trialSetting);
} catch (ex) {
this._log("Failed to get the preferred tweak values from the remote.", LOG_LEVEL_NOTICE);
return {
status: RemotePreferredTweakStatuses.UNAVAILABLE,
error: ex,
};
}
this._log("Failed to connect to the remote server.", LOG_LEVEL_NOTICE);
return false;
}
async _checkAndAskUseRemoteConfiguration(
@@ -281,8 +282,8 @@ export class ModuleResolvingMismatchedTweaks extends AbstractModule {
return { result: false, requireFetch: false };
}
const preferred = await this.services.tweakValue.fetchRemotePreferred(trialSetting);
if (preferred) {
return await this.services.tweakValue.askUseRemoteConfiguration(trialSetting, preferred);
if (preferred.status === RemotePreferredTweakStatuses.AVAILABLE) {
return await this.services.tweakValue.askUseRemoteConfiguration(trialSetting, preferred.values);
}
return { result: false, requireFetch: false };
}

View File

@@ -54,6 +54,36 @@ function createModule(settingsOverride: Partial<typeof DEFAULT_SETTINGS> = {}) {
}
describe("ModuleResolvingMismatchedTweaks", () => {
it("returns an unconfigured remote result without a separate connection preflight", async () => {
const { module, core } = createModule();
const tryConnectRemote = vi.fn(async () => true);
const getRemotePreferredTweakValues = vi.fn(async () => ({
status: "not-configured" as const,
reason: "milestone-missing" as const,
}));
core._services.replicator = {
getNewReplicator: vi.fn(async () => ({ tryConnectRemote, getRemotePreferredTweakValues })),
};
await expect(module._fetchRemotePreferredTweakValues(core.settings)).resolves.toEqual({
status: "not-configured",
reason: "milestone-missing",
});
expect(getRemotePreferredTweakValues).toHaveBeenCalledOnce();
expect(tryConnectRemote).not.toHaveBeenCalled();
});
it("returns unsupported when no replicator implements the remote type", async () => {
const { module, core } = createModule();
core._services.replicator = {
getNewReplicator: vi.fn(async () => undefined),
};
await expect(module._fetchRemotePreferredTweakValues(core.settings)).resolves.toEqual({
status: "unsupported",
});
});
it("should enable and auto-accept compatible mismatches when the preference is undefined", async () => {
const { module, core, askSelectStringDialogue, applyPartial } = createModule({
autoAcceptCompatibleTweak: undefined,

View File

@@ -141,10 +141,11 @@
</Instruction>
<Instruction>
<ExtraItems title={translateMessage("Advanced")}>
<Check
title={translateMessage("Prevent fetching configuration from server")}
bind:value={preventFetchingConfig}
/>
<Check title={translateMessage("Use this device's settings")} bind:value={preventFetchingConfig}>
<InfoNote>
{translateMessage("Skips checking and applying synchronisation settings from the remote.")}
</InfoNote>
</Check>
</ExtraItems>
</Instruction>
<UserDecisions>

View File

@@ -129,7 +129,9 @@
{#if !isP2P}
<Instruction>
<ExtraItems title={msg("Advanced")}>
<Check title={msg("Prevent fetching configuration from server")} bind:value={preventFetchingConfig} />
<Check title={msg("Use this device's settings")} bind:value={preventFetchingConfig}>
<InfoNote>{msg("Skips checking and applying synchronisation settings from the remote.")}</InfoNote>
</Check>
</ExtraItems>
</Instruction>
{/if}

View File

@@ -10,7 +10,7 @@ import {
synchroniseAllFilesBetweenDBandStorage,
type FullScanOptions,
} from "@vrtmrz/livesync-commonlib/compat/serviceFeatures/offlineScanner";
import { adjustSettingToRemoteIfNeeded, processVaultInitialisation } from "./redFlag";
import { adjustSettingToRemoteIfNeeded, cancelScheduledInitialisation, processVaultInitialisation } from "./redFlag";
export const SIMPLE_FETCH_STAGE1_REMOTE_WINS = "Overwrite all with remote files";
export const SIMPLE_FETCH_STAGE1_NEWER_WINS = "Compare time and take newer";
@@ -194,9 +194,7 @@ export async function askAndPerformFastSetupOnScheduledFetchAll(
if (result === "cancelled") {
log("Fetch cancelled by user.", LOG_LEVEL_NOTICE);
clearRememberedSimpleFetchMode(host);
await cleanupFlag();
host.services.appLifecycle.performRestart();
return false;
return await cancelScheduledInitialisation(host, cleanupFlag);
}
if (result === "aborted") {
log("Fetch exited by user.", LOG_LEVEL_NOTICE);
@@ -208,9 +206,14 @@ export async function askAndPerformFastSetupOnScheduledFetchAll(
return undefined; // Let the detailed setup flow handle it.
}
return await processVaultInitialisation(host, log, async () => {
const settings = host.services.setting.currentSettings();
await adjustSettingToRemoteIfNeeded(host, log, { preventFetchingConfig: false }, settings);
const settings = host.services.setting.currentSettings();
if (!(await adjustSettingToRemoteIfNeeded(host, log, { preventFetchingConfig: false }, settings))) {
log("Fetch initialisation cancelled by user.", LOG_LEVEL_NOTICE);
clearRememberedSimpleFetchMode(host);
return await cancelScheduledInitialisation(host, cleanupFlag);
}
const performFastSetup = async () => {
// 1. Perform fast DB fetch (download remote DB content to local DB)
await host.serviceModules.rebuilder.$fetchLocalDBFast(false);
@@ -246,5 +249,6 @@ export async function askAndPerformFastSetupOnScheduledFetchAll(
clearRememberedSimpleFetchMode(host);
log("Simple fetch and scan operation completed.", LOG_LEVEL_NOTICE);
return true;
});
};
return await processVaultInitialisation(host, log, performFastSetup, "keep-on-failure");
}

View File

@@ -10,7 +10,10 @@ import RebuildEverything from "@/modules/features/SetupWizard/dialogs/RebuildEve
import { extractObject } from "octagonal-wheels/object";
import { REMOTE_MINIO, REMOTE_P2P } from "@vrtmrz/livesync-commonlib/compat/common/models/setting.const";
import type { ObsidianLiveSyncSettings } from "@vrtmrz/livesync-commonlib/settings";
import { TweakValuesShouldMatchedTemplate } from "@vrtmrz/livesync-commonlib/compat/common/models/tweak.definition";
import {
RemotePreferredTweakStatuses,
TweakValuesShouldMatchedTemplate,
} from "@vrtmrz/livesync-commonlib/compat/common/models/tweak.definition";
import type {
FetchEverythingResult,
RebuildEverythingResult,
@@ -19,6 +22,7 @@ import { askAndPerformFastSetupOnScheduledFetchAll } from "./redFlag.simpleFetch
import { ConnectionStringParser } from "@vrtmrz/livesync-commonlib/compat/common/ConnectionString";
import { activateRemoteConfiguration } from "@vrtmrz/livesync-commonlib/remote-configurations";
import { isP2PMainRemote } from "@/common/remoteConfiguration";
import { $msg } from "@/common/translation";
/**
* Flag file handler interface, similar to target filter pattern.
@@ -29,6 +33,8 @@ interface FlagFileHandler {
handle: () => Promise<boolean>;
}
type InitialisationOperation = "fetch" | "rebuild";
export async function isFlagFileExist(host: NecessaryServices<never, "storageAccess">, path: string) {
const redFlagExist = await host.serviceModules.storageAccess.isExists(
host.serviceModules.storageAccess.normalisePath(path)
@@ -144,7 +150,7 @@ export function createFetchAllFlagHandler(
// Select the remote database if there are multiple remotes configured.
const isRemoteActivated = await askAndActivateRemoteDatabase(host, log);
if (!isRemoteActivated) {
return false;
return await cancelScheduledInitialisation(host, cleanupFlag);
}
// Ask user for use Fast Setup
@@ -158,9 +164,7 @@ export function createFetchAllFlagHandler(
await host.services.UI.dialogManager.openWithExplicitCancel<FetchEverythingResult>(FetchEverything);
if (method === "cancelled") {
log("Fetch everything cancelled by user.", LOG_LEVEL_NOTICE);
await cleanupFlag();
host.services.appLifecycle.performRestart();
return false;
return await cancelScheduledInitialisation(host, cleanupFlag);
}
const { vault, extra } = method;
const settings = await Promise.resolve(host.services.setting.currentSettings());
@@ -185,9 +189,11 @@ export function createFetchAllFlagHandler(
},
} as const;
if (!(await adjustSettingToRemoteIfNeeded(host, log, extra, settings))) {
log("Fetch initialisation cancelled by user.", LOG_LEVEL_NOTICE);
return await cancelScheduledInitialisation(host, cleanupFlag);
}
return await processVaultInitialisation(host, log, async () => {
const settings = host.services.setting.currentSettings();
await adjustSettingToRemoteIfNeeded(host, log, extra, settings);
const vaultStateToAction = mapVaultStateToAction[vault];
const { makeLocalChunkBeforeSync, makeLocalFilesBeforeSync } = vaultStateToAction;
log(
@@ -217,115 +223,184 @@ export function createFetchAllFlagHandler(
/**
* Adjust setting to remote configuration.
* @param config current configuration to retrieve remote preferred config
* @returns updated configuration if applied, otherwise null.
* @param operation operation which will consume the selected configuration
* @returns whether initialisation may continue.
*/
export async function adjustSettingToRemote(
host: NecessaryServices<"tweakValue" | "UI" | "setting", never>,
log: LogFunction,
config: ObsidianLiveSyncSettings
) {
// Fetch remote configuration unless prevented.
const SKIP_FETCH = "Skip and proceed";
const RETRY_FETCH = "Retry (recommended)";
let canProceed = false;
do {
const remoteTweaks = await host.services.tweakValue.fetchRemotePreferred(config);
if (!remoteTweaks) {
config: ObsidianLiveSyncSettings,
operation: InitialisationOperation = "fetch"
): Promise<boolean> {
while (true) {
const remoteResult = await host.services.tweakValue.fetchRemotePreferred(config);
if (remoteResult.status === RemotePreferredTweakStatuses.NOT_CONFIGURED) {
const useDeviceSettings = $msg("Use this device's settings");
const cancelInitialisation = $msg("Cancel");
log(`Remote synchronisation settings are not configured (${remoteResult.reason}).`, LOG_LEVEL_INFO);
const choice = await host.services.UI.confirm.askSelectStringDialogue(
"Could not fetch configuration from remote. If you are new to the Self-hosted LiveSync, this might be expected. If not, you should check your network or server settings.",
[SKIP_FETCH, RETRY_FETCH] as const,
$msg(
"The selected remote has no saved synchronisation settings. This is normal for a new remote. Use this device's settings, or cancel if you expected existing settings."
),
[useDeviceSettings, cancelInitialisation] as const,
{
defaultAction: RETRY_FETCH,
defaultAction: useDeviceSettings,
timeout: 0,
title: "Fetch Remote Configuration Failed",
title: $msg("No Synchronisation Settings Found"),
}
);
if (choice === SKIP_FETCH) {
canProceed = true;
}
} else {
const necessary = extractObject(TweakValuesShouldMatchedTemplate, remoteTweaks);
// Check if any necessary tweak value is different from current config.
const differentItems = Object.entries(necessary).filter(([key, value]) => {
return config[key as keyof ObsidianLiveSyncSettings] !== value;
});
if (differentItems.length === 0) {
log("Remote configuration matches local configuration. No changes applied.", LOG_LEVEL_NOTICE);
} else {
await host.services.UI.confirm.askSelectStringDialogue(
"Your settings differed slightly from the server's. The plug-in has supplemented the incompatible parts with the server settings!",
["OK"] as const,
return choice === useDeviceSettings;
}
if (remoteResult.status === RemotePreferredTweakStatuses.UNAVAILABLE) {
const retryRemoteSettings = $msg("Retry");
const useDeviceSettings = $msg("Use this device's settings");
const cancelInitialisation = $msg("Cancel");
log("Could not read synchronisation settings from the remote.", LOG_LEVEL_NOTICE);
log(remoteResult.error, LOG_LEVEL_VERBOSE);
if (operation === "rebuild") {
const choice = await host.services.UI.confirm.askSelectStringDialogue(
$msg(
"Could not read the remote's synchronisation settings. Retry, or continue the overwrite with this device's settings. A working connection is still required."
),
[retryRemoteSettings, useDeviceSettings, cancelInitialisation] as const,
{
defaultAction: "OK",
defaultAction: retryRemoteSettings,
timeout: 0,
title: $msg("Could Not Read Synchronisation Settings"),
}
);
if (choice === retryRemoteSettings) continue;
return choice === useDeviceSettings;
}
config = {
...config,
...(Object.fromEntries(differentItems) as Partial<ObsidianLiveSyncSettings>),
} satisfies ObsidianLiveSyncSettings;
await host.services.setting.applyExternalSettings(config, true);
log("Remote configuration applied.", LOG_LEVEL_NOTICE);
canProceed = true;
const updatedConfig = host.services.setting.currentSettings();
return updatedConfig;
const choice = await host.services.UI.confirm.askSelectStringDialogue(
$msg(
"Could not read the remote's synchronisation settings. Check the connection and credentials, then retry."
),
[retryRemoteSettings, cancelInitialisation] as const,
{
defaultAction: retryRemoteSettings,
timeout: 0,
title: $msg("Could Not Read Synchronisation Settings"),
}
);
if (choice === retryRemoteSettings) continue;
return false;
}
} while (!canProceed);
if (remoteResult.status === RemotePreferredTweakStatuses.UNSUPPORTED) {
log("Remote synchronisation settings are not supported by this remote type.", LOG_LEVEL_INFO);
return true;
}
const remoteTweaks = remoteResult.values;
const necessary = extractObject(TweakValuesShouldMatchedTemplate, remoteTweaks);
// Check if any necessary tweak value is different from current config.
const differentItems = Object.entries(necessary).filter(([key, value]) => {
return config[key as keyof ObsidianLiveSyncSettings] !== value;
});
if (differentItems.length === 0) {
log("Remote configuration matches local configuration. No changes applied.", LOG_LEVEL_NOTICE);
} else {
await host.services.UI.confirm.askSelectStringDialogue(
"Your settings differed slightly from the server's. The plug-in has supplemented the incompatible parts with the server settings!",
["OK"] as const,
{
defaultAction: "OK",
timeout: 0,
}
);
}
config = {
...config,
...(Object.fromEntries(differentItems) as Partial<ObsidianLiveSyncSettings>),
} satisfies ObsidianLiveSyncSettings;
await host.services.setting.applyExternalSettings(config, true);
log("Remote configuration applied.", LOG_LEVEL_NOTICE);
return true;
}
}
/**
* Adjust setting to remote if needed.
* @param extra result of dialogues that may contain preventFetchingConfig flag (e.g, from FetchEverything or RebuildEverything)
* @param config current configuration to retrieve remote preferred config
* @param operation operation which will consume the selected configuration
*/
export async function adjustSettingToRemoteIfNeeded(
host: NecessaryServices<"tweakValue" | "UI" | "setting", never>,
log: LogFunction,
extra: { preventFetchingConfig: boolean },
config: ObsidianLiveSyncSettings
) {
if (extra && extra.preventFetchingConfig) {
return;
extra: { preventFetchingConfig: boolean } | null,
config: ObsidianLiveSyncSettings,
operation: InitialisationOperation = "fetch"
): Promise<boolean> {
if (extra?.preventFetchingConfig) {
return true;
}
// P2P has no centralised remote configuration; skip to avoid a spurious
// "Failed to connect to the remote server" error dialog.
if (config.remoteType === REMOTE_P2P) {
log("Remote configuration fetch skipped (P2P mode).", LOG_LEVEL_INFO);
return;
return true;
}
// Remote configuration fetched and applied.
if (await adjustSettingToRemote(host, log, config)) {
config = host.services.setting.currentSettings();
} else {
const canProceed = await adjustSettingToRemote(host, log, config, operation);
if (!canProceed) {
log("Remote configuration not applied.", LOG_LEVEL_NOTICE);
}
// log(JSON.stringify(config), LOG_LEVEL_VERBOSE);
return canProceed;
}
/**
* Process vault initialisation with suspending file watching and sync.
* @param proc process to be executed during initialisation, should return true if can be continued, false if app is unable to continue the process.
* @param keepSuspending whether to keep suspending file watching after the process.
* @returns result of the process, or false if error occurs.
* Cancel a scheduled Fetch or Rebuild without changing the selected automatic
* synchronisation mode. The persisted Scram switches keep both reflection
* directions paused until the existing start-up dialogue resumes them.
*/
export async function cancelScheduledInitialisation(
host: NecessaryServices<"setting" | "appLifecycle", never>,
cleanupFlag: () => Promise<void>
): Promise<false> {
await host.services.setting.applyPartial(
{
suspendFileWatching: true,
suspendParseReplicationResult: true,
},
true
);
await cleanupFlag();
host.services.appLifecycle.performRestart();
return false;
}
type InitialisationSuspensionPolicy = "resume" | "keep" | "keep-on-failure";
/**
* Process Vault initialisation with file watching and synchronisation suspended.
* @param proc Process to execute during initialisation. It returns true only when normal operation may resume.
* @param suspensionPolicy Final file-reflection state. `keep-on-failure` controls both reflection directions so a partly completed Fast Setup remains isolated.
* @returns The result of the process, or false if an error occurs.
*/
export async function processVaultInitialisation(
host: NecessaryServices<"setting", never>,
log: LogFunction,
proc: () => Promise<boolean>,
keepSuspending = false
suspensionPolicy: InitialisationSuspensionPolicy = "resume"
) {
let completed = false;
try {
// Disable batch saving and file watching during initialisation.
await host.services.setting.applyPartial({ batchSave: false }, false);
await host.services.setting.suspendAllSync();
await host.services.setting.suspendExtraSync();
await host.services.setting.applyPartial({ suspendFileWatching: true }, true);
await host.services.setting.applyPartial(
suspensionPolicy === "keep-on-failure"
? { suspendFileWatching: true, suspendParseReplicationResult: true }
: { suspendFileWatching: true },
true
);
try {
const result = await proc();
completed = result;
return result;
} catch (ex) {
log("Error during vault initialisation process.", LOG_LEVEL_NOTICE);
@@ -337,9 +412,21 @@ export async function processVaultInitialisation(
log(ex, LOG_LEVEL_VERBOSE);
return false;
} finally {
if (!keepSuspending) {
// Re-enable file watching after initialisation.
if (suspensionPolicy === "resume") {
await host.services.setting.applyPartial({ suspendFileWatching: false }, true);
} else if (suspensionPolicy === "keep") {
await host.services.setting.applyPartial({ suspendFileWatching: true }, true);
} else {
// Fast Setup owns both directions at this boundary. Reasserting the
// outcome also covers a late failure after finishRebuild started to
// resume reflection, and the legacy doNotSuspendOnFetching path.
await host.services.setting.applyPartial(
{
suspendFileWatching: !completed,
suspendParseReplicationResult: !completed,
},
true
);
}
}
}
@@ -393,12 +480,13 @@ export function createRebuildFlagHandler(
>(RebuildEverything, { isP2P: isP2PMainRemote(settings) });
if (method === "cancelled") {
log("Rebuild everything cancelled by user.", LOG_LEVEL_NOTICE);
await cleanupFlag();
host.services.appLifecycle.performRestart();
return false;
return await cancelScheduledInitialisation(host, cleanupFlag);
}
const { extra } = method;
await adjustSettingToRemoteIfNeeded(host, log, extra, settings);
if (!(await adjustSettingToRemoteIfNeeded(host, log, extra, settings, "rebuild"))) {
log("Rebuild initialisation cancelled by user.", LOG_LEVEL_NOTICE);
return await cancelScheduledInitialisation(host, cleanupFlag);
}
return await processVaultInitialisation(host, log, async () => {
await host.serviceModules.rebuilder.$rebuildEverything();
await cleanupFlag();
@@ -445,7 +533,7 @@ export function createSuspendFlagHandler(
await host.services.setting.applyPartial({ writeLogToTheFile: true }, true);
return Promise.resolve(false);
},
true
"keep"
);
};

View File

@@ -68,6 +68,8 @@ const createLoggerMock = (): LogFunction => {
return vi.fn();
};
const availableRemoteTweaks = (values: Record<string, unknown>) => ({ status: "available", values }) as const;
const createStorageAccessMock = () => {
const files: Set<string> = new Set();
return {
@@ -149,7 +151,9 @@ const createRebuilderMock = () => {
const createTweakValueMock = () => {
return {
fetchRemotePreferred: vi.fn(() => Promise.resolve<any>(null)),
fetchRemotePreferred: vi.fn(() =>
Promise.resolve<any>({ status: "unavailable", error: new Error("Remote unavailable") })
),
};
};
@@ -321,13 +325,13 @@ describe("Red Flag Feature", () => {
() => {
return Promise.resolve(true);
},
false
"resume"
);
expect(host.mocks.setting.currentSettings().suspendFileWatching).toBe(false);
});
it("should keep suspending when keepSuspending is true", async () => {
it("should keep suspending when the policy is keep", async () => {
const host = createHostMock();
const log = createLoggerMock();
@@ -337,7 +341,7 @@ describe("Red Flag Feature", () => {
() => {
return Promise.resolve(true);
},
true
"keep"
);
expect(host.mocks.setting.currentSettings().suspendFileWatching).toBe(true);
@@ -353,7 +357,7 @@ describe("Red Flag Feature", () => {
() => {
throw new Error("Process failed");
},
false
"resume"
);
expect(result).toBe(false);
@@ -459,9 +463,9 @@ describe("Red Flag Feature", () => {
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE1_REMOTE_WINS)
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE2_REMOTE_DELETE_ALL);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce({
batchSave: false,
} as any);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce(
availableRemoteTweaks({ batchSave: false })
);
const handler = createFetchAllFlagHandler(host as any, log);
const result = await handler.handle();
@@ -487,9 +491,9 @@ describe("Red Flag Feature", () => {
backup: "backup_skipped",
extra: { preventFetchingConfig: false },
});
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce({
batchSave: false,
} as any);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce(
availableRemoteTweaks({ batchSave: false })
);
const handler = createFetchAllFlagHandler(host as any, log);
const result = await handler.handle();
@@ -506,9 +510,9 @@ describe("Red Flag Feature", () => {
host.mocks.ui.confirm.confirmWithMessage.mockResolvedValueOnce(false);
const handler = createFetchAllFlagHandler(host as any, log);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce({
batchSave: false,
} as any);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce(
availableRemoteTweaks({ batchSave: false })
);
const result = await handler.handle();
expect(result).toBe(false);
@@ -525,9 +529,9 @@ describe("Red Flag Feature", () => {
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE1_REMOTE_WINS)
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE2_REMOTE_DELETE_ALL);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce({
batchSave: false,
} as any);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce(
availableRemoteTweaks({ batchSave: false })
);
const handler = createFetchAllFlagHandler(host as any, log);
const result = await handler.handle();
@@ -577,6 +581,16 @@ describe("Red Flag Feature", () => {
expect(result).toBe(false);
expect(host.mocks.ui.confirm.confirmWithMessage).not.toHaveBeenCalled();
expect(host.mocks.storageAccess.files.has(FlagFilesOriginal.FETCH_ALL)).toBe(false);
await expect(handler.check()).resolves.toBe(false);
expect(host.mocks.setting.applyPartial).toHaveBeenCalledWith(
{
suspendFileWatching: true,
suspendParseReplicationResult: true,
},
true
);
expect(host.mocks.appLifecycle.performRestart).toHaveBeenCalledOnce();
});
it("should activate selected remote configuration", async () => {
@@ -755,6 +769,79 @@ describe("Red Flag Feature", () => {
});
describe("askAndPerformFastSetupOnScheduledFetchAll", () => {
it("releases both reflection suspensions after Fast Setup succeeds", async () => {
const host = createHostMock();
const log = createLoggerMock();
const cleanupFlag = vi.fn().mockResolvedValue(undefined);
Object.assign(host.mocks.setting.settings, {
doNotSuspendOnFetching: true,
suspendParseReplicationResult: true,
});
host.mocks.ui.confirm.confirmWithMessage
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE1_NEWER_WINS)
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE2_NEWER_CLEANUP);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValue(availableRemoteTweaks({ batchSave: false }));
await expect(askAndPerformFastSetupOnScheduledFetchAll(host as any, log, cleanupFlag)).resolves.toBe(true);
expect(host.mocks.setting.currentSettings()).toMatchObject({
suspendFileWatching: false,
suspendParseReplicationResult: false,
});
});
it("keeps Vault reflection suspended and preserves recovery state when Fast Fetch fails", async () => {
const host = createHostMock();
const log = createLoggerMock();
const cleanupFlag = vi.fn().mockResolvedValue(undefined);
host.mocks.ui.confirm.confirmWithMessage
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE1_NEWER_WINS)
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE2_NEWER_CLEANUP);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValue(availableRemoteTweaks({ batchSave: false }));
host.mocks.rebuilder.$fetchLocalDBFast.mockRejectedValueOnce(new Error("cannot decrypt remote document"));
await expect(askAndPerformFastSetupOnScheduledFetchAll(host as any, log, cleanupFlag)).resolves.toBe(false);
expect(host.mocks.setting.currentSettings()).toMatchObject({
suspendFileWatching: true,
suspendParseReplicationResult: true,
});
expect(host.mocks.rebuilder.finishRebuild).not.toHaveBeenCalled();
expect(cleanupFlag).not.toHaveBeenCalled();
expect(host.mocks.setting.deleteSmallConfig).not.toHaveBeenCalledWith("simple-fetch-mode");
});
it("re-suspends both reflection directions when finalisation fails after releasing them", async () => {
const host = createHostMock();
const log = createLoggerMock();
const cleanupFlag = vi.fn().mockResolvedValue(undefined);
host.mocks.ui.confirm.confirmWithMessage
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE1_NEWER_WINS)
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE2_NEWER_CLEANUP);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValue(availableRemoteTweaks({ batchSave: false }));
host.mocks.rebuilder.finishRebuild.mockImplementationOnce(async () => {
await host.mocks.setting.applyPartial(
{
suspendFileWatching: false,
suspendParseReplicationResult: false,
},
true
);
throw new Error("Vault scan failed after reflection resumed");
});
await expect(askAndPerformFastSetupOnScheduledFetchAll(host as any, log, cleanupFlag)).resolves.toBe(false);
expect(host.mocks.setting.currentSettings()).toMatchObject({
suspendFileWatching: true,
suspendParseReplicationResult: true,
});
expect(cleanupFlag).not.toHaveBeenCalled();
});
it("should remember quick flow choices while the scheduled fetch is pending", async () => {
const host = createHostMock();
const log = createLoggerMock();
@@ -763,7 +850,7 @@ describe("Red Flag Feature", () => {
host.mocks.ui.confirm.confirmWithMessage
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE1_NEWER_WINS)
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE2_NEWER_CLEANUP);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValue({ batchSave: false } as any);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValue(availableRemoteTweaks({ batchSave: false }));
host.mocks.rebuilder.$fetchLocalDBFast.mockRejectedValueOnce(new Error("offline"));
await askAndPerformFastSetupOnScheduledFetchAll(host as any, log, cleanupFlag);
@@ -781,7 +868,7 @@ describe("Red Flag Feature", () => {
host.mocks.ui.confirm.confirmWithMessage
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE1_REMOTE_WINS)
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE2_REMOTE_DELETE_ALL);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValue({ batchSave: false } as any);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValue(availableRemoteTweaks({ batchSave: false }));
await askAndPerformFastSetupOnScheduledFetchAll(host as any, log, cleanupFlag);
@@ -831,6 +918,65 @@ describe("Red Flag Feature", () => {
expect(host.mocks.rebuilder.$fetchLocalDBFast).not.toHaveBeenCalled();
});
it("should preserve automatic synchronisation choices and enter Scram when quick Fetch is cancelled", async () => {
const host = createHostMock();
const cleanupFlag = vi.fn().mockResolvedValue(undefined);
Object.assign(host.mocks.setting.settings, {
liveSync: true,
periodicReplication: true,
syncOnSave: true,
syncOnEditorSave: true,
syncOnStart: true,
syncOnFileOpen: true,
syncAfterMerge: true,
suspendParseReplicationResult: false,
});
host.mocks.ui.confirm.confirmWithMessage
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE1_REMOTE_WINS)
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE2_REMOTE_DELETE_ALL);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce({
status: "not-configured",
reason: "milestone-missing",
});
host.mocks.ui.confirm.askSelectStringDialogue.mockResolvedValueOnce("Cancel");
const result = await askAndPerformFastSetupOnScheduledFetchAll(
host as any,
createLoggerMock(),
cleanupFlag
);
expect(result).toBe(false);
expect(host.mocks.rebuilder.$fetchLocalDBFast).not.toHaveBeenCalled();
expect(host.mocks.setting.suspendAllSync).not.toHaveBeenCalled();
expect(host.mocks.setting.applyPartial).toHaveBeenCalledWith(
{
suspendFileWatching: true,
suspendParseReplicationResult: true,
},
true
);
expect(host.mocks.setting.currentSettings()).toMatchObject({
liveSync: true,
periodicReplication: true,
syncOnSave: true,
syncOnEditorSave: true,
syncOnStart: true,
syncOnFileOpen: true,
syncAfterMerge: true,
suspendFileWatching: true,
suspendParseReplicationResult: true,
});
expect(cleanupFlag).toHaveBeenCalledOnce();
expect(host.mocks.appLifecycle.performRestart).toHaveBeenCalledOnce();
expect(host.mocks.setting.applyPartial.mock.invocationCallOrder[0]).toBeLessThan(
cleanupFlag.mock.invocationCallOrder[0]
);
expect(cleanupFlag.mock.invocationCallOrder[0]).toBeLessThan(
host.mocks.appLifecycle.performRestart.mock.invocationCallOrder[0]
);
});
it("should reboot and return false when sync has failures and user chooses rerun", async () => {
const host = createHostMock();
const log = createLoggerMock();
@@ -839,7 +985,9 @@ describe("Red Flag Feature", () => {
host.mocks.ui.confirm.confirmWithMessage
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE1_REMOTE_WINS)
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE2_REMOTE_DELETE_ALL);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce({ batchSave: false } as any);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce(
availableRemoteTweaks({ batchSave: false })
);
(synchroniseAllFilesBetweenDBandStorage as any).mockResolvedValueOnce(false);
host.mocks.ui.confirm.askSelectStringDialogue.mockResolvedValueOnce("Reboot to re-run the process");
@@ -859,7 +1007,9 @@ describe("Red Flag Feature", () => {
host.mocks.ui.confirm.confirmWithMessage
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE1_REMOTE_WINS)
.mockResolvedValueOnce(SIMPLE_FETCH_STAGE2_REMOTE_DELETE_ALL);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce({ batchSave: false } as any);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce(
availableRemoteTweaks({ batchSave: false })
);
(synchroniseAllFilesBetweenDBandStorage as any).mockResolvedValueOnce(false);
host.mocks.ui.confirm.askSelectStringDialogue.mockResolvedValueOnce(
"Finalise the process and resume normal operation"
@@ -1017,9 +1167,9 @@ describe("Red Flag Feature", () => {
const host = createHostMock();
const config = { batchSave: true } as any;
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce({
batchSave: false,
} as any);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce(
availableRemoteTweaks({ batchSave: false })
);
await adjustSettingToRemoteIfNeeded(
host as any,
@@ -1047,7 +1197,9 @@ describe("Red Flag Feature", () => {
const differentConfig = {
[key]: differentValue,
};
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce(differentConfig as any);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce(
availableRemoteTweaks(differentConfig)
);
host.mocks.ui.confirm.askSelectStringDialogue.mockResolvedValueOnce("OK");
await adjustSettingToRemote(host as any, createLoggerMock(), config);
@@ -1074,7 +1226,9 @@ describe("Red Flag Feature", () => {
const differentConfig = {
[key]: differentValue,
};
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce(differentConfig as any);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce(
availableRemoteTweaks(differentConfig)
);
host.mocks.ui.confirm.askSelectStringDialogue.mockResolvedValueOnce("OK");
await adjustSettingToRemote(host as any, createLoggerMock(), config);
@@ -1084,32 +1238,71 @@ describe("Red Flag Feature", () => {
}
);
it("should show dialog when remote fetch fails", async () => {
it("should explain that missing remote settings are normal for a new database without offering retry", async () => {
const host = createHostMock();
const log = createLoggerMock();
const config = { batchSave: true } as any;
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce(null);
host.mocks.ui.confirm.askSelectStringDialogue.mockResolvedValueOnce("Skip and proceed");
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce({
status: "not-configured",
reason: "milestone-missing",
});
host.mocks.ui.confirm.askSelectStringDialogue.mockResolvedValueOnce("Use this device's settings");
await adjustSettingToRemote(host as any, log, config);
await expect(adjustSettingToRemote(host as any, log, config)).resolves.toBe(true);
expect(host.mocks.ui.confirm.askSelectStringDialogue).toHaveBeenCalled();
expect(host.mocks.ui.confirm.askSelectStringDialogue).toHaveBeenCalledWith(
"The selected remote has no saved synchronisation settings. This is normal for a new remote. Use this device's settings, or cancel if you expected existing settings.",
["Use this device's settings", "Cancel"],
{
defaultAction: "Use this device's settings",
timeout: 0,
title: "No Synchronisation Settings Found",
}
);
expect(host.mocks.tweakValue.fetchRemotePreferred).toHaveBeenCalledOnce();
expect(host.mocks.setting.applyExternalSettings).not.toHaveBeenCalled();
});
it("should retry when user selects retry option", async () => {
it("should retry only when remote settings are unavailable", async () => {
const host = createHostMock();
const log = createLoggerMock();
const config = { batchSave: true } as any;
const failure = new Error("network failed");
host.mocks.tweakValue.fetchRemotePreferred
.mockResolvedValueOnce(null)
.mockResolvedValueOnce({ batchSave: false } as any);
host.mocks.ui.confirm.askSelectStringDialogue.mockResolvedValueOnce("Retry (recommended)");
.mockResolvedValueOnce({ status: "unavailable", error: failure })
.mockResolvedValueOnce({ status: "available", values: { batchSave: false } } as any);
host.mocks.ui.confirm.askSelectStringDialogue.mockResolvedValueOnce("Retry");
await adjustSettingToRemote(host as any, log, config);
await expect(adjustSettingToRemote(host as any, log, config)).resolves.toBe(true);
expect(host.mocks.tweakValue.fetchRemotePreferred).toHaveBeenCalledTimes(2);
expect(host.mocks.ui.confirm.askSelectStringDialogue).toHaveBeenCalledWith(
"Could not read the remote's synchronisation settings. Check the connection and credentials, then retry.",
["Retry", "Cancel"],
{
defaultAction: "Retry",
timeout: 0,
title: "Could Not Read Synchronisation Settings",
}
);
});
it("should cancel initialisation instead of proceeding after an unavailable remote", async () => {
const host = createHostMock();
const config = { batchSave: true } as any;
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce({
status: "unavailable",
error: new Error("network failed"),
});
host.mocks.ui.confirm.askSelectStringDialogue.mockResolvedValueOnce("Cancel");
await expect(adjustSettingToRemote(host as any, createLoggerMock(), config)).resolves.toBe(false);
expect(host.mocks.tweakValue.fetchRemotePreferred).toHaveBeenCalledOnce();
expect(host.mocks.setting.applyExternalSettings).not.toHaveBeenCalled();
});
it("should log when no changes needed", async () => {
@@ -1117,9 +1310,9 @@ describe("Red Flag Feature", () => {
const log = createLoggerMock();
const config = { batchSave: false } as any;
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce({
batchSave: false,
} as any);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce(
availableRemoteTweaks({ batchSave: false })
);
await adjustSettingToRemote(host as any, log, config);
@@ -1131,8 +1324,11 @@ describe("Red Flag Feature", () => {
const log = createLoggerMock();
const config = { batchSave: true } as any;
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce(null);
host.mocks.ui.confirm.askSelectStringDialogue.mockResolvedValueOnce("Skip and proceed");
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce({
status: "not-configured",
reason: "preferred-values-missing",
});
host.mocks.ui.confirm.askSelectStringDialogue.mockResolvedValueOnce("Use this device's settings");
await adjustSettingToRemoteIfNeeded(host as any, log, null as any, config);
@@ -1239,7 +1435,7 @@ describe("Red Flag Feature", () => {
() => {
return Promise.resolve(false);
},
true
"keep"
);
expect(host.mocks.setting.currentSettings().suspendFileWatching).toBe(true);
@@ -1460,6 +1656,93 @@ describe("Red Flag Feature", () => {
});
describe("flagHandlerToEventHandler integration", () => {
it("should stop a detailed Fetch when remote-setting initialisation is cancelled", async () => {
const host = createHostMock();
host.mocks.storageAccess.files.add(FlagFilesOriginal.FETCH_ALL);
host.mocks.ui.confirm.confirmWithMessage.mockResolvedValueOnce(SIMPLE_FETCH_STAGE1_DETAILED);
host.mocks.ui.dialogManager.openWithExplicitCancel.mockResolvedValueOnce({
vault: "independent",
extra: { preventFetchingConfig: false },
});
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce({
status: "not-configured",
reason: "milestone-missing",
});
host.mocks.ui.confirm.askSelectStringDialogue.mockResolvedValueOnce("Cancel");
const result = await createFetchAllFlagHandler(host as any, createLoggerMock()).handle();
expect(result).toBe(false);
expect(host.mocks.rebuilder.$fetchLocal).not.toHaveBeenCalled();
expect(host.mocks.setting.suspendAllSync).not.toHaveBeenCalled();
expect(host.mocks.setting.applyPartial).toHaveBeenCalledWith(
{
suspendFileWatching: true,
suspendParseReplicationResult: true,
},
true
);
expect(host.mocks.storageAccess.files.has(FlagFilesOriginal.FETCH_ALL)).toBe(false);
expect(host.mocks.appLifecycle.performRestart).toHaveBeenCalledOnce();
});
it("should stop Rebuild before deleting local data when remote-setting initialisation is cancelled", async () => {
const host = createHostMock();
host.mocks.storageAccess.files.add(FlagFilesOriginal.REBUILD_ALL);
host.mocks.ui.dialogManager.openWithExplicitCancel.mockResolvedValueOnce({
extra: { preventFetchingConfig: false },
});
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce({
status: "not-configured",
reason: "milestone-missing",
});
host.mocks.ui.confirm.askSelectStringDialogue.mockResolvedValueOnce("Cancel");
const result = await createRebuildFlagHandler(host as any, createLoggerMock()).handle();
expect(result).toBe(false);
expect(host.mocks.rebuilder.$rebuildEverything).not.toHaveBeenCalled();
expect(host.mocks.setting.suspendAllSync).not.toHaveBeenCalled();
expect(host.mocks.setting.applyPartial).toHaveBeenCalledWith(
{
suspendFileWatching: true,
suspendParseReplicationResult: true,
},
true
);
expect(host.mocks.storageAccess.files.has(FlagFilesOriginal.REBUILD_ALL)).toBe(false);
expect(host.mocks.appLifecycle.performRestart).toHaveBeenCalledOnce();
});
it("should let Rebuild use this device's settings when remote settings are unavailable", async () => {
const host = createHostMock();
host.mocks.storageAccess.files.add(FlagFilesOriginal.REBUILD_ALL);
host.mocks.ui.dialogManager.openWithExplicitCancel.mockResolvedValueOnce({
extra: { preventFetchingConfig: false },
});
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce({
status: "unavailable",
error: new Error("network failed"),
});
host.mocks.ui.confirm.askSelectStringDialogue.mockResolvedValueOnce("Use this device's settings");
const result = await createRebuildFlagHandler(host as any, createLoggerMock()).handle();
expect(result).toBe(true);
expect(host.mocks.ui.confirm.askSelectStringDialogue).toHaveBeenCalledWith(
"Could not read the remote's synchronisation settings. Retry, or continue the overwrite with this device's settings. A working connection is still required.",
["Retry", "Use this device's settings", "Cancel"],
{
defaultAction: "Retry",
timeout: 0,
title: "Could Not Read Synchronisation Settings",
}
);
expect(host.mocks.rebuilder.$rebuildEverything).toHaveBeenCalledOnce();
expect(host.mocks.storageAccess.files.has(FlagFilesOriginal.REBUILD_ALL)).toBe(false);
expect(host.mocks.appLifecycle.performRestart).not.toHaveBeenCalled();
});
it("should return true when flag does not exist", async () => {
const host = createHostMock();
const log = createLoggerMock();
@@ -1476,7 +1759,7 @@ describe("Red Flag Feature", () => {
const log = createLoggerMock();
host.mocks.storageAccess.files.add(FlagFilesOriginal.FETCH_ALL);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce({});
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce(availableRemoteTweaks({}));
host.mocks.ui.confirm.confirmWithMessage.mockResolvedValueOnce(SIMPLE_FETCH_STAGE1_DETAILED);
host.mocks.ui.dialogManager.openWithExplicitCancel.mockResolvedValueOnce("cancelled");
@@ -1554,9 +1837,7 @@ describe("Red Flag Feature", () => {
it("should handle fetchAll flag with flagHandlerToEventHandler identical", async () => {
const host = createHostMock();
const log = createLoggerMock();
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValue({
customChunkSize: 1,
} as any);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValue(availableRemoteTweaks({ customChunkSize: 1 }));
host.mocks.storageAccess.files.add(FlagFilesOriginal.FETCH_ALL);
host.mocks.ui.confirm.confirmWithMessage.mockResolvedValueOnce(SIMPLE_FETCH_STAGE1_DETAILED);
@@ -1574,9 +1855,9 @@ describe("Red Flag Feature", () => {
it("should handle rebuildAll flag with flagHandlerToEventHandler", async () => {
const host = createHostMock();
const log = createLoggerMock();
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce({
customChunkSize: 1,
} as any);
host.mocks.tweakValue.fetchRemotePreferred.mockResolvedValueOnce(
availableRemoteTweaks({ customChunkSize: 1 })
);
host.mocks.storageAccess.files.add(FlagFilesOriginal.REBUILD_ALL);
host.mocks.ui.dialogManager.openWithExplicitCancel.mockResolvedValueOnce({ extra: {} });

View File

@@ -30,12 +30,15 @@ Deno.test({
const aggregator = await browser.newPage();
const assertNoAggregatorFailures = observePageFailures(aggregator);
const assertNoAggregatorNetworkFailures = observeNetworkFailures(aggregator);
await aggregator.goto(new URL("aggregator.html#id=pages-smoke&n=2&i=0&d=first-", server.baseUrl).href);
await aggregator.goto(new URL("aggregator.html#id=pages-smoke&n=2&i=0&d=before%2", server.baseUrl).href);
await aggregator.getByText("1 / 2 Loaded", { exact: true }).waitFor();
await aggregator.goto(new URL("aggregator.html#id=pages-smoke&n=2&i=1&d=second", server.baseUrl).href);
await aggregator.goto(
new URL("aggregator.html#id=pages-smoke&n=2&i=1&d=3after%26amp%2Bplus%25percent", server.baseUrl)
.href
);
assertEquals(
await aggregator.getByRole("link", { name: "Open Obsidian to complete setup" }).getAttribute("href"),
"obsidian://setuplivesync?settingsQR=first-second"
"obsidian://setuplivesync?settingsQR=before%23after%26amp%2Bplus%25percent"
);
assertNoAggregatorFailures();
assertNoAggregatorNetworkFailures();

View File

@@ -254,17 +254,23 @@ export async function confirmRebuild(port: number, captures: SetupCaptureNames):
return screenshot;
}
export async function skipMissingRemoteConfiguration(port: number, captures: SetupCaptureNames): Promise<string> {
const title = "Fetch Remote Configuration Failed";
export async function continueWithoutRemoteSettings(port: number, captures: SetupCaptureNames): Promise<string> {
const title = "No Synchronisation Settings Found";
const screenshot = await captureGuideDialogue(
port,
`guide-${captures.guide}-missing-remote-configuration.png`,
title
);
await withObsidianPage(port, async (page) => {
await modalByTitle(page, title)
.getByRole("button", { name: "Skip and proceed" })
.click({ timeout: uiTimeoutMs });
const modal = modalByTitle(page, title);
await modal
.getByText("This is normal for a new remote.", { exact: false })
.waitFor({ state: "visible", timeout: uiTimeoutMs });
await modal.getByRole("button", { name: "Cancel", exact: true }).waitFor({
state: "visible",
timeout: uiTimeoutMs,
});
await modal.getByRole("button", { name: "Use this device's settings" }).click({ timeout: uiTimeoutMs });
});
return screenshot;
}

View File

@@ -25,7 +25,7 @@ import {
modalByTitle,
resumeCompatibilityReviewIfShown,
selectRadioOption,
skipMissingRemoteConfiguration,
continueWithoutRemoteSettings,
type SetupArtifact,
} from "../runner/setupUri.ts";
import { captureObsidianPage, withObsidianPage } from "../runner/ui.ts";
@@ -278,7 +278,7 @@ async function main(): Promise<void> {
screenshots.push(...(await enterManualCouchDBSettings(session.remoteDebuggingPort, couchDb, dbName)));
screenshots.push(await captureAndStartInitialisation(session.remoteDebuggingPort, "new", captures));
screenshots.push(await confirmRebuild(session.remoteDebuggingPort, captures));
screenshots.push(await skipMissingRemoteConfiguration(session.remoteDebuggingPort, captures));
screenshots.push(await continueWithoutRemoteSettings(session.remoteDebuggingPort, captures));
screenshots.push(await acknowledgeDisabledOptionalFeatures(session.remoteDebuggingPort, captures));
const state = await finishInitialisation(session.remoteDebuggingPort, context.cliBinary, session.cliEnv);
await resumeCompatibilityReviewIfShown(session.remoteDebuggingPort);

View File

@@ -29,7 +29,7 @@ import {
finishInitialisation,
generateSetupURIFromDevice,
resumeCompatibilityReviewIfShown,
skipMissingRemoteConfiguration,
continueWithoutRemoteSettings,
type SetupArtifact,
type SetupCaptureNames,
} from "../runner/setupUri.ts";
@@ -226,7 +226,7 @@ async function main(): Promise<void> {
screenshots.push(await enterSetupURI(portA, "new", bootstrapArtifact, captures));
screenshots.push(await captureAndStartInitialisation(portA, "new", captures));
screenshots.push(await confirmRebuild(portA, captures));
screenshots.push(await skipMissingRemoteConfiguration(portA, captures));
screenshots.push(await continueWithoutRemoteSettings(portA, captures));
screenshots.push(await acknowledgeDisabledOptionalFeatures(portA, captures));
const firstState = await finishInitialisation(portA, context.cliBinary, sessionA.cliEnv);
await resumeCompatibilityReviewIfShown(portA);

View File

@@ -300,8 +300,8 @@ async function confirmRebuild(port: number): Promise<string> {
return screenshot;
}
async function skipMissingRemoteConfiguration(port: number): Promise<string> {
const title = "Fetch Remote Configuration Failed";
async function continueWithoutRemoteSettings(port: number): Promise<string> {
const title = "No Synchronisation Settings Found";
const screenshot = await captureObsidianDialogue(
port,
"setup-uri-first-missing-remote-configuration.png",
@@ -309,16 +309,20 @@ async function skipMissingRemoteConfiguration(port: number): Promise<string> {
const modal = modalByTitle(page, title);
await modal.waitFor({ state: "visible", timeout: uiTimeoutMs });
await modal
.getByText("If you are new to the Self-hosted LiveSync, this might be expected.", {
.getByText("This is normal for a new remote.", {
exact: false,
})
.waitFor({ state: "visible", timeout: uiTimeoutMs });
await modal.getByRole("button", { name: "Cancel", exact: true }).waitFor({
state: "visible",
timeout: uiTimeoutMs,
});
}
);
await captureGuideDialogue(port, "guide-quick-setup-missing-remote-configuration.png", title);
await withObsidianPage(port, async (page) => {
await modalByTitle(page, title)
.getByRole("button", { name: "Skip and proceed" })
.getByRole("button", { name: "Use this device's settings" })
.click({ timeout: uiTimeoutMs });
});
return screenshot;
@@ -736,7 +740,7 @@ async function main(): Promise<void> {
await enterSetupURI(session.remoteDebuggingPort, "new", artifact);
screenshots.push(await captureAndStartInitialisation(session.remoteDebuggingPort, "new"));
screenshots.push(await confirmRebuild(session.remoteDebuggingPort));
screenshots.push(await skipMissingRemoteConfiguration(session.remoteDebuggingPort));
screenshots.push(await continueWithoutRemoteSettings(session.remoteDebuggingPort));
screenshots.push(await acknowledgeDisabledOptionalFeatures(session.remoteDebuggingPort));
const firstCompletion = await finishInitialisation(
session.remoteDebuggingPort,

View File

@@ -12,6 +12,54 @@ Earlier releases remain available in the 0.25 release history and the legacy rel
## Unreleased
## 1.0.9
8th August, 2026
For the first time in a while, I published a release that could not be promoted to a stable release. Sorry about that! I am glad that we caught it while it was still a pre-release.
### Setup and compatibility
#### Fixed
- Multi-part settings QR codes now preserve special characters in passwords, passphrases, and other settings (PR #1083). Thank you to @calvinbui for the improvement!
- Fast Setup now sizes each finite CouchDB changes page from a one-row status probe, counts the returned result together with `pending`, and resumes from the page's opaque `last_seq` without comparing token representations. Each page uses a one-second idle timeout instead of a heartbeat, allowing CouchDB 3.2 to return its terminator after the currently available rows have been persisted.
## 1.0.8
8th August, 2026
This version was published for pre-release validation only and was not promoted to a stable release.
### Setup and compatibility
#### Fixed
- Fast Setup now sizes each finite CouchDB changes page from a one-row status probe, counts the returned result together with `pending`, and resumes from the page's opaque `last_seq` without comparing token representations. Heartbeat-enabled feeds no longer wait for future writes after the currently available rows have been persisted (#1065).
- Cancelling remote selection during a scheduled Fetch now removes the Fetch flag before restarting with file and database reflection paused, preventing the same selection dialogue from reopening on every start-up.
## 1.0.7
8th August, 2026
### Setup and compatibility
#### Fixed
- Fast Setup now completes only after the captured CouchDB changes target has been persisted. Decryption, protocol, and local write failures stop the operation without finalising an incomplete database, while transient interruptions resume from the last durable checkpoint (#1065).
## 1.0.6
6th August, 2026
I know that onboarding, and other parts which feel unclear or confusing, still need improvement. Please do report any such cases.
### Setup and compatibility
#### Fixed
- Initial setup now distinguishes an empty remote with no saved synchronisation settings from a failed remote read. New remotes can use this device's settings without an unnecessary retry; Fetch pauses on unreadable settings, while Rebuild can explicitly continue with this device's settings. Cancelling preserves the selected automatic synchronisation mode and restarts with Vault and database reflection paused (#1064). Thank you to @mateus2k2 for the follow-up report!
## 1.0.5
5th August, 2026

View File

@@ -17,5 +17,9 @@
"1.0.2": "1.7.2",
"1.0.3": "1.7.2",
"1.0.4": "1.7.2",
"1.0.5": "1.7.2"
"1.0.5": "1.7.2",
"1.0.6": "1.7.2",
"1.0.7": "1.7.2",
"1.0.8": "1.7.2",
"1.0.9": "1.7.2"
}