Compare commits

..

23 Commits

Author SHA1 Message Date
vorotamoroz
236f9c434e Add P2P signalling-only netem benchmark 2026-07-08 10:04:03 +00:00
vorotamoroz
708bd187d8 Add P2P split-container netem stress fixture 2026-07-08 10:03:35 +00:00
vorotamoroz
46081a5f47 Record P2P selected candidate paths 2026-07-08 10:03:11 +00:00
vorotamoroz
e5aa8d5c88 Add CouchDB netem shim benchmark 2026-07-08 10:02:41 +00:00
vorotamoroz
abae4bda28 Add netem smoke fixture for benchmark simulation 2026-07-08 07:38:54 +00:00
vorotamoroz
dc9b4fd37e Document CLI benchmark comparison model 2026-07-08 07:03:23 +00:00
vorotamoroz
cb1fb0f874 Lower benchmark relay file descriptor limit 2026-07-08 06:54:42 +00:00
vorotamoroz
2cca2355fd Stabilise CLI P2P smoke readiness 2026-07-08 06:48:17 +00:00
vorotamoroz
5e090a3971 Run CLI network smoke on pull requests 2026-07-08 06:35:26 +00:00
vorotamoroz
fd251346e3 Add Compose-based CLI network benchmarks 2026-07-08 06:30:12 +00:00
vorotamoroz
6a9918677f Merge pull request #996 from vrtmrz/0_25_80
releasing 0.25.80
2026-07-07 21:21:34 +09:00
vorotamoroz
2d42b92a89 bump 2026-07-07 12:15:58 +00:00
vorotamoroz
a2b794c520 Merge pull request #995 from vrtmrz/fix_conflict_issues
Fix conflict issues
2026-07-07 21:13:00 +09:00
vorotamoroz
6f9446f447 Merge remote-tracking branch 'origin/main' into fix_conflict_issues 2026-07-07 12:11:38 +00:00
vorotamoroz
3f9cd67b1c Merge pull request #992 from vrtmrz/fix_989
fix: enable hidden file sync before overwrite setup
2026-07-07 21:10:09 +09:00
vorotamoroz
dbcbf2c5ca fix: document safer conflict preservation 2026-07-07 12:05:40 +00:00
vorotamoroz
eed0fca8d3 fix: preserve ambiguous conflict candidates 2026-07-07 12:00:24 +00:00
vorotamoroz
05e031b90b update submodule pointer 2026-07-07 11:31:29 +00:00
vorotamoroz
bec767c13f add notes 2026-07-07 11:30:36 +00:00
vorotamoroz
8046a777af fix: refine conflict merge policy 2026-07-07 11:29:51 +00:00
vorotamoroz
0c58b0c513 test: reproduce conflict issue reports 2026-07-07 11:07:09 +00:00
vorotamoroz
b66e227a02 update doc
update submodule pointer
2026-07-07 10:54:49 +00:00
vorotamoroz
af6df84b5d fix: enable hidden file sync before overwrite setup 2026-07-03 05:09:55 +00:00
47 changed files with 2469 additions and 771 deletions

View File

@@ -16,7 +16,9 @@ pouchdb-browser.js
production/
# Test coverage and reports
coverage/
coverage/
test/bench-network/bench-results/
src/apps/cli/testdeno/bench-results/
# Local environment / secrets
.env

View File

@@ -0,0 +1,84 @@
# Run the Compose-packaged CLI P2P smoke benchmark.
#
# This workflow is intentionally non-required at first. It exercises the local
# Compose package for CouchDB + Nostr relay + CLI runner, and uploads the
# benchmark JSON results for inspection.
name: cli-p2p-compose-smoke
on:
pull_request:
paths:
- '.github/workflows/cli-p2p-compose-smoke.yml'
- 'package.json'
- 'package-lock.json'
- 'src/apps/cli/**'
- 'test/bench-network/**'
workflow_dispatch:
inputs:
cases:
description: 'Comma-separated benchmark cases'
required: false
default: 'couchdb-baseline,p2p-direct-local'
md_files:
description: 'Markdown file count'
required: false
default: '2'
bin_files:
description: 'Binary file count'
required: false
default: '1'
couchdb_rtt_ms:
description: 'Requested CouchDB RTT in milliseconds'
required: false
default: '20'
permissions:
contents: read
jobs:
smoke:
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- name: Checkout
uses: actions/checkout@v4
with:
submodules: recursive
- name: Show Docker versions
run: |
docker --version
docker compose version
- name: Run Compose P2P smoke benchmark
env:
BENCH_CASES: ${{ inputs.cases || 'couchdb-baseline,p2p-direct-local' }}
BENCH_MD_FILE_COUNT: ${{ inputs.md_files || '2' }}
BENCH_MD_MIN_SIZE_BYTES: '128'
BENCH_MD_MAX_SIZE_BYTES: '256'
BENCH_BIN_FILE_COUNT: ${{ inputs.bin_files || '1' }}
BENCH_BIN_SIZE_BYTES: '512'
BENCH_COUCHDB_RTT_MS: ${{ inputs.couchdb_rtt_ms || '20' }}
BENCH_SYNC_TIMEOUT: '180'
BENCH_PEERS_TIMEOUT: '20'
LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS: '60000'
BENCH_LIVESYNC_TEST_TEE: '0'
run: docker compose -f test/bench-network/compose.yml run --rm bench-runner
- name: Show Compose diagnostics
if: failure()
run: |
docker compose -f test/bench-network/compose.yml ps
docker compose -f test/bench-network/compose.yml logs --no-color couchdb nostr-relay
- name: Upload benchmark results
if: always()
uses: actions/upload-artifact@v4
with:
name: cli-p2p-compose-smoke-results
path: test/bench-network/bench-results/**
if-no-files-found: warn
- name: Stop Compose services
if: always()
run: docker compose -f test/bench-network/compose.yml down -v --remove-orphans

14
devs.md
View File

@@ -148,6 +148,20 @@ Hence, the new feature should be implemented as follows:
- **Service Hub** (`src/modules/services/`): Central service registry using dependency injection
- **Common Library** (`src/lib/`): Platform-independent sync logic, shared with other tools
### Conflict Merge Policy
Markdown conflict auto-merge should behave like a conservative three-way merge. The guiding rule is to merge changes when they touch non-overlapping regions, and to keep a manual conflict when the edits overlap semantically.
When in doubt, prefer the safer outcome: preserve data, keep the conflict visible, and ask the user rather than silently discarding content or choosing one side.
- If one side deletes a line and the other side leaves that same line unchanged, treat it as a safe deletion. The deleted line must not be reintroduced into the merged result.
- If one side inserts new content in a different region while the other side deletes an unchanged old region, preserve the insertion and the deletion.
- If one side deletes a line and the other side modifies that same line, keep the conflict for user resolution.
- If both sides insert different content at the same position, keep both insertions in a deterministic order unless the surrounding deletion context indicates that they are competing replacements.
- Avoid resolving conflicts by simply choosing the newest revision unless the user has explicitly selected that behaviour.
This policy is intentionally aligned with the conflict checkboxes and compatibility settings: automatic merge should remove avoidable prompts, but it must not silently choose between overlapping user intentions.
### File Structure Conventions
- **Platform-specific code**: Use `.platform.ts` suffix (replaced with `.obsidian.ts` in production builds via esbuild)

View File

@@ -1,360 +0,0 @@
# Architectural Decision Record: Setting Definition Repository
## Status
Proposed
## Release
Not scheduled. Intended as a design direction before refactoring the settings dialogue.
## Context
The current settings dialogue is implemented around `ObsidianLiveSyncSettingTab`.
It is effective and feature rich, but several responsibilities are combined in the
same layer:
- editing-state buffering and dirty-state tracking,
- local-only dialogue state such as `configPassphrase`, `preset`, `syncMode`, and
`deviceAndVaultName`,
- persistence through `SettingService`,
- Obsidian `Setting` component rendering,
- pane layout and visibility rules,
- validation and value coercion,
- setup wizard transitions,
- rebuild/restart side effects,
- remote diagnostics and maintenance actions.
Some setting metadata already exists in `configurationNames` and related setting
constants. This is a useful seed, but it is not a complete source of truth. The
metadata does not currently describe storage domain, value kind, validation,
capability requirements, migration behaviour, cross-setting dependencies, or
whether a value should be rendered by a generic control or a custom pane.
The settings dialogue also contains a mix of simple controls and workflow panels.
Examples:
- Simple controls: `showStatusOnEditor`, `syncOnSave`, `customChunkSize`,
`readChunksOnline`, `useTimeouts`.
- Derived or dialogue-only values: `syncMode`, `preset`, `configPassphrase`,
`deviceAndVaultName`.
- Workflow panels: remote configuration management, E2EE setup, setup wizard,
local/remote rebuild, maintenance commands, Customisation Sync dialogue open.
This matters primarily for maintainability and platform independence. A setting
should have one shared definition of its domain meaning regardless of whether it
is displayed in Obsidian, surfaced in a CLI, used by a WebApp, or documented.
Tests benefit from this separation, but testability is a consequence rather than
the main design goal.
Real Obsidian E2E remains the right signal for Obsidian shell behaviour such as
opening the settings tab, rendering real Obsidian components, and verifying
user-visible workflows. Harness-based tests can then focus on deterministic
setting semantics that no longer depend on mounting the whole Obsidian setting
tab.
## Decision
Introduce a platform-neutral Setting Definition Repository as the source of
truth for setting metadata and setting semantics.
The repository should live in the shared domain layer, not under the Obsidian
dialogue implementation and not under a generic model bucket. In the current
layout this means `src/lib/src/common/settings`.
The repository should not be an Obsidian UI abstraction. It should describe what
a setting is and how it behaves. Obsidian, CLI, WebApp, tests, and documentation
can then consume the same definitions through their own renderers or adapters.
The Obsidian settings tab should use an Obsidian renderer that maps repository
definitions to native Obsidian `Setting` components for simple controls, while
workflow panes remain custom.
The existing Obsidian setting dialogue should be migrated incrementally. Complex
workflow panes should remain custom-rendered at first. Simple controls should
move to repository-driven rendering first.
## Proposed Model
Each setting definition should describe a single setting key or a dialogue-only
virtual key.
```ts
type SettingStorageDomain = "persisted" | "local" | "derived" | "ephemeral";
type SettingValueKind = "boolean" | "text" | "password" | "number" | "select" | "textarea" | "string-list" | "custom";
type SettingCapability = "database-user" | "server-admin" | "filesystem" | "obsidian-shell" | "obsidian-plugin-host";
type SettingDefinition<TSettings, TKey extends keyof TSettings | string> = {
key: TKey;
storage: SettingStorageDomain;
kind: SettingValueKind;
defaultValue?: unknown;
labelKey: string;
descriptionKey?: string;
label: string;
description?: string;
category: string;
pane?: string;
section?: string;
level?: "ADVANCED" | "POWER_USER" | "EDGE_CASE";
status?: "BETA" | "ALPHA" | "EXPERIMENTAL";
obsolete?: boolean;
internal?: boolean;
placeholder?: string;
options?: Record<string, string>;
secret?: boolean;
requiredCapabilities?: SettingCapability[];
visible?: (context: SettingEvaluationContext<TSettings>) => boolean;
enabled?: (context: SettingEvaluationContext<TSettings>) => boolean;
validate?: (value: unknown, context: SettingEvaluationContext<TSettings>) => SettingValidationResult;
coerce?: (value: unknown, context: SettingEvaluationContext<TSettings>) => unknown;
affects?: SettingEffect[];
commit?: SettingCommitPolicy<TKey>;
render?: "auto" | "custom";
};
```
`SettingEvaluationContext` should carry the current editing settings, persisted
settings, platform capabilities, and remote capability information if known. It
should not carry an Obsidian `App`.
`labelKey` and `descriptionKey` should be i18n keys. During migration, the key
may be the literal English text already used by `configurationNames` and
`SettingInformation`. This matches the current i18n behaviour where an unknown
key resolves to the key itself, avoids adding translation resources up front, and
lets us later replace literal keys with stable resource keys without changing
consumers. `label` and `description` remain compatibility aliases while existing
code still expects resolved strings.
`internal` should mark settings that are not currently editable from the UI.
Obsolete settings are internal by default. Missing UI metadata alone should not
make a setting internal; `kind`, `render`, and explicit `internal` metadata
should decide whether the automatic renderer can safely handle it.
`commit` should describe when a value is persisted, not how a button is rendered.
Immediate settings can save on change. Explicit settings are held in the editing
buffer until an apply action commits the configured group. This keeps apply
buttons out of the repository while still making grouped save behaviour
testable.
## Storage Domains
Settings should be classified by where they live:
- `persisted`: normal `ObsidianLiveSyncSettings` values saved through
`SettingService`.
- `local`: values stored outside the main settings document, for example local
storage or device/vault identity.
- `derived`: values computed from persisted settings, for example `syncMode`.
- `ephemeral`: dialogue-only inputs such as `preset`.
This makes current special cases explicit:
- `configPassphrase` is local-only.
- `deviceAndVaultName` is local/service managed.
- `syncMode` is derived from `liveSync` and `periodicReplication`.
- `preset` is ephemeral and expands to several persisted settings.
## Rendering Strategy
The repository should support generic rendering, but it should not force every
pane to become schema-driven immediately.
Use three levels:
1. **Auto-rendered controls**
Simple `boolean`, `number`, `text`, `password`, `select`, and `textarea`
settings. These can replace many `new Setting(...).autoWire...` calls.
2. **Repository-defined groups with custom sections**
A pane can declare layout, headings, and order through the repository but keep
a custom renderer for the section body.
3. **Fully custom workflow panes**
Remote configuration management, E2EE setup, setup wizard, maintenance, and
rebuild flows should remain custom until their side effects are separately
modelled.
The Obsidian setting dialogue becomes a renderer of repository definitions plus a
host for custom workflow panes.
## Side Effects
Setting changes should distinguish value persistence from effects.
Examples of effects:
- `requires-local-rebuild`
- `requires-remote-rebuild`
- `requires-restart`
- `requires-apply-settings`
- `suspends-sync`
- `updates-unresolved-error-ui`
- `changes-active-remote`
- `expands-preset`
The current `isNeedRebuildLocal()` and `isNeedRebuildRemote()` methods should
eventually be replaced by repository metadata. This would make rebuild prompts
testable without rendering the full settings tab.
## Capability Requirements
Some settings and actions require capabilities that not all users or platforms
have.
Examples:
- CouchDB server diagnostics and automatic CouchDB repair require server-admin
capability.
- Normal CouchDB sync requires only database-user capability.
- Hidden File Sync and Customisation Sync require filesystem capability.
- Obsidian plug-in reload requires obsidian-plugin-host capability.
- Opening settings panes and workspace views requires obsidian-shell capability.
Capability metadata should be used for:
- warning text in Obsidian settings,
- disabling unsupported actions,
- CLI/WebApp help output,
- Harness tests for visibility and enabled-state rules.
The repository should not introduce a generic cross-platform `PluginManager`
concept. Obsidian plug-in host behaviour should remain an Obsidian-specific
adapter or custom workflow.
## Current Assumptions to Preserve
- Settings can be edited in a buffer before being saved.
- Some values save immediately unless `holdValue` is set.
- Some values require explicit Apply buttons.
- Visibility and enabled-state often depend on other editing values.
- Some settings are hidden in setup wizard mode.
- Advanced, power-user, and edge-case levels remain supported.
- The dialogue can be reloaded while preserving dirty local edits.
- Existing `SettingService` remains responsible for encryption, persistence,
migration, and applying settings.
- Existing complex setup and remote configuration workflows remain custom.
## Migration Plan
### Phase 1: Repository Skeleton
- Create a repository module in the shared setting domain
(`src/lib/src/common/settings`), not under the Obsidian dialogue folder.
- Move existing `configurationNames` metadata into repository definitions without
changing runtime behaviour.
- Add storage domain, kind, i18n keys, internal marker, pane, section, level, and
secret metadata for a small subset of settings.
- Keep `getConfig()`, `getConfName()`, and existing callers working through a
compatibility facade.
### Phase 2: Evaluation API
- Add pure functions:
- `getSettingDefinition(key)`
- `listSettingDefinitions(filter)`
- `evaluateSetting(definition, context)`
- `validateSettingValue(key, value, context)`
- `getSettingEffects(changedKeys, context)`
- Add unit tests for derived values, visibility, enabled-state, validation, and
rebuild/restart effects.
### Phase 3: Obsidian Renderer for Simple Controls
- Add a small renderer that maps repository definitions to Obsidian `Setting`
controls.
- Migrate one low-risk pane first, likely Appearance/Logging or Advanced memory
cache settings.
- Keep custom panes untouched.
- Keep `LiveSyncSetting` as a compatibility wrapper during migration.
### Phase 4: Derived and Local Values
- Model `syncMode`, `preset`, `configPassphrase`, and `deviceAndVaultName`
explicitly.
- Replace ad hoc save paths in `ObsidianLiveSyncSettingTab` with storage-domain
handlers.
- Keep user-visible behaviour unchanged.
### Phase 5: Effects and Capability Warnings
- Replace `isNeedRebuildLocal()` and `isNeedRebuildRemote()` with
repository-driven effect calculation.
- Model explicit commit groups for settings that must be applied together, for
example configuration encryption passphrase settings, setting sync file, and
database suffix changes.
- Add capability metadata for CouchDB diagnostics, repair, Hidden File Sync, and
Obsidian-only plug-in operations.
- Use this to improve warnings for database-scoped CouchDB users and
administrator-only actions.
### Phase 6: Documentation and Non-Obsidian Consumers
- Treat documentation as an authored source, not as an output that must be fully
generated from code.
- Optionally combine repository metadata with a documentation source such as YAML
to generate or lint `docs/settings.md`.
- Use the repository to verify that documented settings exist, that defaults and
storage domains are consistent, and that internal settings are intentionally
omitted or documented as internal.
- Expose repository metadata to CLI/WebApp where useful.
- Let Harness tests assert the same repository semantics used by Obsidian.
## Testing Strategy
Use Harness or unit tests for:
- default value coverage,
- type/kind consistency,
- every persisted setting has a definition or is explicitly internal,
- visibility and enabled-state predicates,
- derived values such as `syncMode`,
- preset expansion,
- rebuild/restart effect calculation,
- capability warnings.
Use real Obsidian E2E for:
- opening the actual setting tab,
- rendering Obsidian `Setting` components,
- setup wizard flow,
- remote configuration workflow,
- actual restart prompts,
- workflows that depend on Obsidian settings shell behaviour.
## Consequences
Positive:
- Setting semantics are maintained in one platform-neutral place.
- Setting semantics become testable without mounting Obsidian UI.
- Documentation, CLI, WebApp, and Obsidian can share setting metadata where it is
useful.
- Capability-sensitive settings become explicit.
- Future settings are less likely to be implemented in only one surface.
- The Obsidian settings dialogue can be refactored incrementally.
Negative:
- There will be a temporary compatibility layer between old setting constants and
the repository.
- Some panes will remain custom, so the repository will not remove all UI code.
- Definition metadata can become stale if not enforced by tests.
- Over-generalising workflow panes would make the repository harder to maintain.
## Non-Goals
- Do not replace `SettingService` persistence in the first phase.
- Do not make Obsidian plug-in host operations cross-platform.
- Do not convert all setting panes to schema-driven UI at once.
- Do not require real Obsidian E2E for every setting definition.
- Do not remove custom renderers for remote setup, E2EE setup, or maintenance
workflows.
## Open Questions
- What should the exact `CapabilityProvider` interface look like for static
platform capabilities and runtime-probed remote capabilities? This should be
decided while implementing the Obsidian renderer so the interface follows a
real consumer instead of an abstract capability model.

View File

@@ -1,7 +1,7 @@
{
"id": "obsidian-livesync",
"name": "Self-hosted LiveSync",
"version": "0.25.79",
"version": "0.25.80",
"minAppVersion": "1.7.2",
"description": "Community implementation of self-hosted livesync. Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"author": "vorotamoroz",

10
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{
"name": "obsidian-livesync",
"version": "0.25.79",
"version": "0.25.80",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "obsidian-livesync",
"version": "0.25.79",
"version": "0.25.80",
"license": "MIT",
"workspaces": [
"src/apps/cli",
@@ -16210,7 +16210,7 @@
},
"src/apps/cli": {
"name": "self-hosted-livesync-cli",
"version": "0.25.79-cli",
"version": "0.25.80-cli",
"dependencies": {
"chokidar": "^4.0.0",
"minimatch": "^10.2.5",
@@ -16236,7 +16236,7 @@
},
"src/apps/webapp": {
"name": "livesync-webapp",
"version": "0.25.79-webapp",
"version": "0.25.80-webapp",
"dependencies": {
"octagonal-wheels": "^0.1.47"
},
@@ -16251,7 +16251,7 @@
}
},
"src/apps/webpeer": {
"version": "0.25.79-webpeer",
"version": "0.25.80-webpeer",
"dependencies": {
"octagonal-wheels": "^0.1.47"
},

View File

@@ -1,6 +1,6 @@
{
"name": "obsidian-livesync",
"version": "0.25.79",
"version": "0.25.80",
"description": "Reflect your vault changes to some other devices immediately. Please make sure to disable other synchronize solutions to avoid content corruption or duplication.",
"main": "main.js",
"type": "module",

View File

@@ -4,12 +4,21 @@ import type { ServiceContext } from "@lib/services/base/ServiceBase";
import { LiveSyncTrysteroReplicator } from "@lib/replication/trystero/LiveSyncTrysteroReplicator";
import { compatGlobal } from "@lib/common/coreEnvFunctions.ts";
import { LiveSyncError } from "@lib/common/LSError";
import { getPeerConnectionStats } from "@lib/rpc/transports/DiagRTCPeerConnections.utils";
import { appendFile } from "node:fs/promises";
type CLIP2PPeer = {
peerId: string;
name: string;
};
type CandidateSummary = {
id: string | "unknown";
candidateType: string | "unknown";
protocol: string | "unknown";
relayProtocol: string | "unknown";
};
function delay(ms: number): Promise<void> {
return new Promise((resolve) => compatGlobal.setTimeout(resolve, ms));
}
@@ -81,6 +90,74 @@ function resolvePeer(peers: CLIP2PPeer[], peerToken: string): CLIP2PPeer | undef
return undefined;
}
function getReportValue<T extends string | number>(
report: Record<string, unknown> | undefined,
key: string
): T | "unknown" {
const value = report?.[key];
return typeof value === "string" || typeof value === "number" ? (value as T) : "unknown";
}
function summariseCandidate(reports: unknown[], candidateId: string | "unknown"): CandidateSummary | undefined {
if (candidateId === "unknown") {
return undefined;
}
const report = reports.map((r) => r as Record<string, unknown>).find((r) => r.id === candidateId);
if (!report) {
return undefined;
}
return {
id: candidateId,
candidateType: getReportValue<string>(report, "candidateType"),
protocol: getReportValue<string>(report, "protocol"),
relayProtocol: getReportValue<string>(report, "relayProtocol"),
};
}
async function writePeerConnectionStatsIfRequested(
replicator: LiveSyncTrysteroReplicator,
peer: CLIP2PPeer
): Promise<void> {
const outputPath = process.env.LIVESYNC_P2P_STATS_JSONL?.trim();
if (!outputPath) {
return;
}
const peerConnection = replicator.rawHost?.room?.getPeers()[peer.peerId];
const stats = peerConnection ? await getPeerConnectionStats(`cli-p2p-${peer.peerId}`, peerConnection) : undefined;
const localCandidate = summariseCandidate(stats?.reports ?? [], stats?.localCandidateId ?? "unknown");
const remoteCandidate = summariseCandidate(stats?.reports ?? [], stats?.remoteCandidateId ?? "unknown");
const selectedPath =
localCandidate && remoteCandidate
? `${localCandidate.candidateType}<->${remoteCandidate.candidateType}`
: "unknown";
const payload = {
generatedAt: new Date().toISOString(),
command: "p2p-sync",
peerId: peer.peerId,
peerName: peer.name,
candidatePathCollected: !!stats?.selectedPair,
selectedPath,
selectedPair: stats
? {
id: stats.selectedPairId,
state: stats.state,
currentRoundTripTime: stats.currentRoundTripTime,
totalRoundTripTime: stats.totalRoundTripTime,
requestsSent: stats.requestsSent,
responsesReceived: stats.responsesReceived,
packetsDiscardedOnSend: stats.packetsDiscardedOnSend,
bytesSent: stats.bytesSent,
bytesReceived: stats.bytesReceived,
}
: undefined,
localCandidate,
remoteCandidate,
};
await appendFile(outputPath, `${JSON.stringify(payload)}\n`, "utf8");
}
export async function syncWithPeer(
core: LiveSyncBaseCore<ServiceContext, never>,
peerToken: string,
@@ -118,6 +195,7 @@ export async function syncWithPeer(
: LiveSyncError.fromError(err ?? "P2P sync failed while requesting remote sync");
}
await writePeerConnectionStatsIfRequested(replicator, targetPeer);
return targetPeer;
} finally {
await replicator.close();

View File

@@ -1,7 +1,7 @@
{
"name": "self-hosted-livesync-cli",
"private": true,
"version": "0.25.79-cli",
"version": "0.25.80-cli",
"main": "dist/index.cjs",
"type": "module",
"scripts": {

View File

@@ -1,10 +1,11 @@
import { TempDir } from "./helpers/temp.ts";
import { applyRemoteSyncSettings, initSettingsFile } from "./helpers/settings.ts";
import { assertFilesEqual, runCliOrFail } from "./helpers/cli.ts";
import { startCouchdb, stopCouchdb } from "./helpers/docker.ts";
import { createCouchdbDatabase, startCouchdb, stopCouchdb } from "./helpers/docker.ts";
import { createDeterministicDataset, type DatasetEntry } from "./helpers/dataset.ts";
type BenchmarkConfig = {
caseName: string;
couchdbBackendUri: string;
couchdbProxyUri: string;
couchdbUser: string;
@@ -21,6 +22,10 @@ type BenchmarkConfig = {
requestedRttMs: number;
passphrase: string;
encrypt: boolean;
managedCouchdb: boolean;
simulationTier: string;
networkProfile: string;
networkModel: string;
};
function readEnvString(name: string, fallback: string): string {
@@ -70,6 +75,7 @@ function formatBytes(value: number): string {
function buildConfig(): BenchmarkConfig {
return {
caseName: readEnvString("BENCH_CASE", "couchdb-baseline"),
couchdbBackendUri: readEnvString("BENCH_COUCHDB_BACKEND_URI", "http://127.0.0.1:5989"),
couchdbProxyUri: readEnvString("BENCH_COUCHDB_URI", "http://127.0.0.1:15989"),
couchdbUser: readEnvString("BENCH_COUCHDB_USER", readEnvString("username", "admin")),
@@ -86,6 +92,10 @@ function buildConfig(): BenchmarkConfig {
requestedRttMs: Math.floor(readEnvNumber("BENCH_COUCHDB_RTT_MS", 50)),
passphrase: readEnvString("BENCH_PASSPHRASE", `bench-${Date.now()}`),
encrypt: readEnvBool("BENCH_ENCRYPT", true),
managedCouchdb: readEnvBool("BENCH_COUCHDB_MANAGED", true),
simulationTier: readEnvString("BENCH_SIMULATION_TIER", "1"),
networkProfile: readEnvString("BENCH_NETWORK_PROFILE", "http-latency-proxy"),
networkModel: readEnvString("BENCH_NETWORK_MODEL", "local-http-proxy"),
};
}
@@ -200,7 +210,17 @@ async function main(): Promise<void> {
await initSettingsFile(settingsA);
await initSettingsFile(settingsB);
await startCouchdb(config.couchdbBackendUri, config.couchdbUser, config.couchdbPassword, config.couchdbDbname);
if (config.managedCouchdb) {
await startCouchdb(config.couchdbBackendUri, config.couchdbUser, config.couchdbPassword, config.couchdbDbname);
} else {
console.log(`[INFO] using externally managed CouchDB: ${config.couchdbBackendUri}`);
await createCouchdbDatabase(
config.couchdbBackendUri,
config.couchdbUser,
config.couchdbPassword,
config.couchdbDbname
);
}
const proxy = startCouchdbProxy({
backendUri: config.couchdbBackendUri,
@@ -265,10 +285,15 @@ async function main(): Promise<void> {
}
const result = {
caseName: config.caseName,
mode: "couchdb-cli-benchmark",
couchdbBackendUri: config.couchdbBackendUri,
couchdbProxyUri: config.couchdbProxyUri,
couchdbDbname: config.couchdbDbname,
managedCouchdb: config.managedCouchdb,
simulationTier: config.simulationTier,
networkProfile: config.networkProfile,
networkModel: config.networkModel,
rttRequestedMs: config.requestedRttMs,
proxyApplied: proxy.applied,
proxyNote: proxy.note,
@@ -300,7 +325,9 @@ async function main(): Promise<void> {
);
} finally {
await proxy.stop();
await stopCouchdb().catch(() => {});
if (config.managedCouchdb) {
await stopCouchdb().catch(() => {});
}
}
}

View File

@@ -0,0 +1,133 @@
type SweepResult = {
name: string;
runner: "p2p" | "couchdb";
rttMs?: number;
result: Record<string, unknown>;
};
function readEnvString(name: string, fallback: string): string {
const value = Deno.env.get(name)?.trim();
return value && value.length > 0 ? value : fallback;
}
function timestamp(): string {
const d = new Date();
const pad = (n: number) => String(n).padStart(2, "0");
return (
`${d.getUTCFullYear()}${pad(d.getUTCMonth() + 1)}${pad(d.getUTCDate())}-` +
`${pad(d.getUTCHours())}${pad(d.getUTCMinutes())}${pad(d.getUTCSeconds())}`
);
}
function parseRttList(raw: string): number[] {
const values = raw
.split(",")
.map((value) => Number(value.trim()))
.filter((value) => Number.isFinite(value) && value > 0)
.map((value) => Math.floor(value));
if (values.length === 0) {
throw new Error(`BENCH_SWEEP_RTT_MS must contain at least one positive number, got '${raw}'`);
}
return values;
}
function buildBaseEnv(): Record<string, string> {
return {
BENCH_MD_FILE_COUNT: readEnvString("BENCH_MD_FILE_COUNT", "20"),
BENCH_MD_MIN_SIZE_BYTES: readEnvString("BENCH_MD_MIN_SIZE_BYTES", "512"),
BENCH_MD_MAX_SIZE_BYTES: readEnvString("BENCH_MD_MAX_SIZE_BYTES", "2048"),
BENCH_BIN_FILE_COUNT: readEnvString("BENCH_BIN_FILE_COUNT", "5"),
BENCH_BIN_SIZE_BYTES: readEnvString("BENCH_BIN_SIZE_BYTES", "8192"),
BENCH_SYNC_TIMEOUT: readEnvString("BENCH_SYNC_TIMEOUT", "300"),
BENCH_PEERS_TIMEOUT: readEnvString("BENCH_PEERS_TIMEOUT", "60"),
BENCH_SEED: readEnvString("BENCH_SEED", "livesync-benchmark-seed"),
LIVESYNC_TEST_TEE: readEnvString("BENCH_LIVESYNC_TEST_TEE", "0"),
};
}
async function runBenchmark(options: {
taskName: "bench:p2p" | "bench:couchdb";
name: string;
outputDir: string;
env: Record<string, string>;
}): Promise<Record<string, unknown>> {
const resultPath = `${options.outputDir}/${options.name}.json`;
const env = {
...Deno.env.toObject(),
...options.env,
BENCH_RESULT_JSON: resultPath,
};
console.log(`[latency-sweep] running ${options.name}`);
const child = new Deno.Command("deno", {
args: ["task", options.taskName],
cwd: import.meta.dirname,
env,
stdin: "null",
stdout: "inherit",
stderr: "inherit",
}).spawn();
const status = await child.status;
if (status.code !== 0) {
throw new Error(`benchmark failed: ${options.name} (exit ${status.code})`);
}
return JSON.parse(await Deno.readTextFile(resultPath)) as Record<string, unknown>;
}
async function main(): Promise<void> {
const outRoot = readEnvString("BENCH_SWEEP_ROOT", `${import.meta.dirname}/bench-results`);
const outputDir = `${outRoot}/latency-sweep-${timestamp()}`;
const rtts = parseRttList(readEnvString("BENCH_SWEEP_RTT_MS", "20,50,100,150,300"));
const base = buildBaseEnv();
await Deno.mkdir(outputDir, { recursive: true });
const results: SweepResult[] = [];
if (readEnvString("BENCH_SWEEP_INCLUDE_P2P", "true") !== "false") {
const p2pResult = await runBenchmark({
taskName: "bench:p2p",
name: "p2p-direct-local",
outputDir,
env: {
...base,
BENCH_CASE: "p2p-direct-local",
BENCH_TURN_SERVERS: "",
},
});
results.push({ name: "p2p-direct-local", runner: "p2p", result: p2pResult });
}
for (const rtt of rtts) {
const name = `couchdb-rtt-${rtt}ms`;
const couchdbResult = await runBenchmark({
taskName: "bench:couchdb",
name,
outputDir,
env: {
...base,
BENCH_CASE: name,
BENCH_COUCHDB_RTT_MS: String(rtt),
},
});
results.push({ name, runner: "couchdb", rttMs: rtt, result: couchdbResult });
}
const summary = {
generatedAt: new Date().toISOString(),
outputDir,
note:
"This sweep models additional remote CouchDB request latency through the existing HTTP proxy. It is not a full netem model of jitter, loss, MTU, bandwidth, or VPN encapsulation.",
rtts,
results,
};
await Deno.writeTextFile(`${outputDir}/summary.json`, JSON.stringify(summary, null, 2));
console.log(JSON.stringify(summary, null, 2));
console.log(`[latency-sweep] result directory: ${outputDir}`);
}
if (import.meta.main) {
main().catch((error) => {
console.error("[Fatal Error]", error);
Deno.exit(1);
});
}

View File

@@ -0,0 +1,312 @@
type BenchmarkCase = {
name: string;
runner: "p2p" | "couchdb";
description: string;
dataPath: string;
trustBoundary: string;
env: Record<string, string>;
};
function readEnvString(name: string, fallback: string): string {
const value = Deno.env.get(name)?.trim();
return value && value.length > 0 ? value : fallback;
}
function readEnvInteger(name: string, fallback: number): number {
const value = readEnvString(name, String(fallback));
const parsed = Number(value);
if (!Number.isInteger(parsed) || parsed < 1) {
throw new Error(`${name} must be a positive integer, got '${value}'`);
}
return parsed;
}
function timestamp(): string {
const d = new Date();
const pad = (n: number) => String(n).padStart(2, "0");
return (
`${d.getUTCFullYear()}${pad(d.getUTCMonth() + 1)}${pad(d.getUTCDate())}-` +
`${pad(d.getUTCHours())}${pad(d.getUTCMinutes())}${pad(d.getUTCSeconds())}`
);
}
function buildBaseEnv(): Record<string, string> {
return {
BENCH_MD_FILE_COUNT: readEnvString("BENCH_MD_FILE_COUNT", "20"),
BENCH_MD_MIN_SIZE_BYTES: readEnvString("BENCH_MD_MIN_SIZE_BYTES", "512"),
BENCH_MD_MAX_SIZE_BYTES: readEnvString("BENCH_MD_MAX_SIZE_BYTES", "2048"),
BENCH_BIN_FILE_COUNT: readEnvString("BENCH_BIN_FILE_COUNT", "5"),
BENCH_BIN_SIZE_BYTES: readEnvString("BENCH_BIN_SIZE_BYTES", "8192"),
BENCH_SYNC_TIMEOUT: readEnvString("BENCH_SYNC_TIMEOUT", "300"),
BENCH_PEERS_TIMEOUT: readEnvString("BENCH_PEERS_TIMEOUT", "60"),
BENCH_SEED: readEnvString("BENCH_SEED", "livesync-benchmark-seed"),
LIVESYNC_TEST_TEE: readEnvString("BENCH_LIVESYNC_TEST_TEE", "0"),
};
}
function buildCases(): BenchmarkCase[] {
const base = buildBaseEnv();
const couchdbRtt = readEnvString("BENCH_COUCHDB_RTT_MS", "20");
const tetheringVpnRtt = readEnvString("BENCH_TETHERING_VPN_RTT_MS", "120");
const localTurnServers = readEnvString("BENCH_LOCAL_TURN_SERVERS", "turn:127.0.0.1:3478");
const shimCouchdbUri = readEnvString("BENCH_SHIM_COUCHDB_URI", "http://couchdb-shim:5984");
const signallingShimRelay = readEnvString("BENCH_SIGNAL_SHIM_RELAY", "ws://p2p-signalling-shim:7777/");
return [
{
name: "couchdb-baseline",
runner: "couchdb",
description: "Standard self-hosted CouchDB path through a local latency proxy.",
dataPath: "Device A -> CouchDB -> Device B",
trustBoundary: "CouchDB operator and network path",
env: {
...base,
BENCH_CASE: "couchdb-baseline",
BENCH_COUCHDB_RTT_MS: couchdbRtt,
},
},
{
name: "p2p-direct-local",
runner: "p2p",
description: "Preferred direct WebRTC P2P path with Nostr signalling and TURN disabled.",
dataPath: "Device A -> Device B",
trustBoundary: "Nostr relay for signalling metadata; no TURN relay",
env: {
...base,
BENCH_CASE: "p2p-direct-local",
BENCH_TURN_SERVERS: "",
BENCH_SIMULATION_TIER: "1",
BENCH_NETWORK_PROFILE: "local-direct",
BENCH_NETWORK_MODEL: "local-runner-webrtc",
BENCH_P2P_CANDIDATE_PATH_VERIFICATION: "turn-disabled-but-selected-ice-pair-not-collected",
},
},
{
name: "couchdb-tethering-vpn-proxy",
runner: "couchdb",
description:
"Approximate smartphone tethering/VPN remote-database path using an HTTP latency proxy. This does not model loss, jitter, MTU, or VPN encapsulation.",
dataPath: "Device A -> VPN/network path -> CouchDB -> VPN/network path -> Device B",
trustBoundary: "VPN/network path and CouchDB operator",
env: {
...base,
BENCH_CASE: "couchdb-tethering-vpn-proxy",
BENCH_COUCHDB_RTT_MS: tetheringVpnRtt,
},
},
{
name: "couchdb-netem-home-wifi",
runner: "couchdb",
description:
"Tier 2 CouchDB path through the Compose netem TCP shim using the home-wifi profile.",
dataPath: "Device A -> netem TCP shim -> CouchDB -> netem TCP shim -> Device B",
trustBoundary: "CouchDB operator and constrained network shim",
env: {
...base,
BENCH_CASE: "couchdb-netem-home-wifi",
BENCH_COUCHDB_BACKEND_URI: shimCouchdbUri,
BENCH_COUCHDB_RTT_MS: "1",
BENCH_SIMULATION_TIER: "2",
BENCH_NETWORK_PROFILE: "home-wifi",
BENCH_NETWORK_MODEL: "compose-netem-tcp-shim",
},
},
{
name: "couchdb-netem-tethering-vpn",
runner: "couchdb",
description:
"Tier 2 CouchDB path through the Compose netem TCP shim using a tethering-vpn profile.",
dataPath: "Device A -> netem TCP shim -> CouchDB -> netem TCP shim -> Device B",
trustBoundary: "CouchDB operator and constrained smartphone/VPN-like network shim",
env: {
...base,
BENCH_CASE: "couchdb-netem-tethering-vpn",
BENCH_COUCHDB_BACKEND_URI: shimCouchdbUri,
BENCH_COUCHDB_RTT_MS: "1",
BENCH_SIMULATION_TIER: "2",
BENCH_NETWORK_PROFILE: "tethering-vpn",
BENCH_NETWORK_MODEL: "compose-netem-tcp-shim",
},
},
{
name: "p2p-smartphone-vpn-direct",
runner: "p2p",
description:
"Direct P2P case name for smartphone tethering/VPN measurements. In this local runner it is unshaped and should be treated as a wiring check unless executed on that network.",
dataPath: "Device A -> Device B when WebRTC direct connectivity succeeds",
trustBoundary: "Smartphone/VPN routing policy plus Nostr signalling metadata",
env: {
...base,
BENCH_CASE: "p2p-smartphone-vpn-direct",
BENCH_TURN_SERVERS: "",
BENCH_SIMULATION_TIER: "unmeasured",
BENCH_NETWORK_PROFILE: "smartphone-vpn-direct-placeholder",
BENCH_NETWORK_MODEL: "local-runner-no-netem",
BENCH_P2P_CANDIDATE_PATH_VERIFICATION:
"structural-placeholder-only; selected ICE pair may be collected, but the path is not shaped",
},
},
{
name: "p2p-signalling-netem-home-wifi",
runner: "p2p",
description:
"Tier 2 P2P path with only the Nostr signalling relay accessed through the home-wifi netem shim.",
dataPath: "Device A -> Device B over WebRTC DataChannel; Nostr signalling through netem shim",
trustBoundary: "Nostr signalling metadata through constrained network shim; no TURN relay",
env: {
...base,
BENCH_CASE: "p2p-signalling-netem-home-wifi",
BENCH_RELAY: signallingShimRelay,
BENCH_TURN_SERVERS: "",
BENCH_SIMULATION_TIER: "2",
BENCH_NETWORK_PROFILE: "home-wifi",
BENCH_NETWORK_MODEL: "compose-netem-signalling-shim",
BENCH_P2P_CANDIDATE_PATH_VERIFICATION:
"selected ICE pair collected; only Nostr signalling path is shaped",
},
},
{
name: "p2p-signalling-netem-tethering-vpn",
runner: "p2p",
description:
"Tier 2 P2P path with only the Nostr signalling relay accessed through the tethering-vpn netem shim.",
dataPath: "Device A -> Device B over WebRTC DataChannel; Nostr signalling through netem shim",
trustBoundary: "Nostr signalling metadata through constrained smartphone/VPN-like network shim; no TURN relay",
env: {
...base,
BENCH_CASE: "p2p-signalling-netem-tethering-vpn",
BENCH_RELAY: signallingShimRelay,
BENCH_TURN_SERVERS: "",
BENCH_SIMULATION_TIER: "2",
BENCH_NETWORK_PROFILE: "tethering-vpn",
BENCH_NETWORK_MODEL: "compose-netem-signalling-shim",
BENCH_P2P_CANDIDATE_PATH_VERIFICATION:
"selected ICE pair collected; only Nostr signalling path is shaped",
},
},
{
name: "p2p-user-turn",
runner: "p2p",
description: "Optional fallback path through a local user-controlled TURN server.",
dataPath: "Device A -> user-controlled TURN -> Device B",
trustBoundary: "User-controlled TURN server",
env: {
...base,
BENCH_CASE: "p2p-user-turn",
BENCH_TURN_SERVERS: localTurnServers,
BENCH_SIMULATION_TIER: "1",
BENCH_NETWORK_PROFILE: "local-turn-fallback",
BENCH_NETWORK_MODEL: "local-runner-webrtc-turn-configured",
BENCH_P2P_CANDIDATE_PATH_VERIFICATION:
"turn-configured; selected ICE pair may still be direct or relayed, so interpret the recorded candidate types",
},
},
];
}
async function runCase(
testCase: BenchmarkCase,
outputDir: string,
repeatIndex: number,
repeatCount: number
): Promise<Record<string, unknown>> {
const suffix = repeatCount > 1 ? `-r${String(repeatIndex).padStart(2, "0")}` : "";
const resultPath = `${outputDir}/${testCase.name}${suffix}.json`;
const taskName = testCase.runner === "p2p" ? "bench:p2p" : "bench:couchdb";
const env = {
...Deno.env.toObject(),
...testCase.env,
BENCH_RESULT_JSON: resultPath,
BENCH_REPEAT_INDEX: String(repeatIndex),
BENCH_REPEAT_COUNT: String(repeatCount),
};
const repeatLabel = repeatCount > 1 ? ` (${repeatIndex}/${repeatCount})` : "";
console.log(`[bench-cases] running ${testCase.name}${repeatLabel}: ${testCase.description}`);
const command = new Deno.Command("deno", {
args: ["task", taskName],
cwd: import.meta.dirname,
env,
stdin: "null",
stdout: "inherit",
stderr: "inherit",
});
const child = command.spawn();
const status = await child.status;
if (status.code !== 0) {
throw new Error(`case failed: ${testCase.name} (exit ${status.code})`);
}
const result = JSON.parse(await Deno.readTextFile(resultPath)) as Record<string, unknown>;
return {
...testCase,
repeatIndex,
repeatCount,
resultPath,
result,
};
}
function selectCases(allCases: BenchmarkCase[]): BenchmarkCase[] {
const requested = readEnvString("BENCH_CASES", "couchdb-baseline,p2p-direct-local");
const names = requested
.split(",")
.map((v) => v.trim())
.filter((v) => v.length > 0);
const byName = new Map(allCases.map((c) => [c.name, c]));
return names.map((name) => {
const found = byName.get(name);
if (!found) {
throw new Error(`Unknown BENCH_CASES entry '${name}'. Available: ${allCases.map((c) => c.name).join(", ")}`);
}
return found;
});
}
async function main(): Promise<void> {
const outRoot = readEnvString("BENCH_CASES_ROOT", `${import.meta.dirname}/bench-results`);
const outputDir = `${outRoot}/cases-${timestamp()}`;
await Deno.mkdir(outputDir, { recursive: true });
const allCases = buildCases();
const cases = selectCases(allCases);
const repeatCount = readEnvInteger("BENCH_REPEAT_COUNT", 1);
await Deno.writeTextFile(
`${outputDir}/case-manifest.json`,
JSON.stringify(
{
generatedAt: new Date().toISOString(),
repeatCount,
selectedCases: cases,
availableCases: allCases,
},
null,
2
)
);
const results: Record<string, unknown>[] = [];
for (const testCase of cases) {
for (let repeatIndex = 1; repeatIndex <= repeatCount; repeatIndex++) {
results.push(await runCase(testCase, outputDir, repeatIndex, repeatCount));
}
}
const summary = {
generatedAt: new Date().toISOString(),
outputDir,
repeatCount,
results,
};
await Deno.writeTextFile(`${outputDir}/summary.json`, JSON.stringify(summary, null, 2));
console.log(JSON.stringify(summary, null, 2));
console.log(`[bench-cases] result directory: ${outputDir}`);
}
if (import.meta.main) {
main().catch((error) => {
console.error("[Fatal Error]", error);
Deno.exit(1);
});
}

View File

@@ -0,0 +1,435 @@
import { join } from "@std/path";
import { startCliInBackground } from "./helpers/backgroundCli.ts";
import { assertFilesEqual, runCliOrFail } from "./helpers/cli.ts";
import { createDeterministicDataset, type DatasetEntry } from "./helpers/dataset.ts";
import { discoverPeer } from "./helpers/p2p.ts";
import { applyP2pSettings, applyP2pTestTweaks, initSettingsFile } from "./helpers/settings.ts";
type Role = "host" | "client";
type NetemSummary = {
enabled: boolean;
profile: string;
interface: string;
delayMs: number;
jitterMs: number;
lossPercent: number;
bandwidthMbit: number;
mtu: number;
tcQdisc?: string;
ipAddr?: string;
ipRoute?: string;
};
type HostReady = {
generatedAt: string;
totalFiles: number;
totalBytes: number;
mdFileCount: number;
binFileCount: number;
mirrorElapsedMs: number;
netem: NetemSummary;
};
type P2PConnectionStats = {
candidatePathCollected: boolean;
selectedPath: string;
localCandidate?: { candidateType: string; protocol: string; relayProtocol: string };
remoteCandidate?: { candidateType: string; protocol: string; relayProtocol: string };
};
function errorToRecord(error: unknown): Record<string, unknown> {
if (error instanceof Error) {
return {
name: error.name,
message: error.message,
stack: error.stack,
};
}
return {
name: "UnknownError",
message: String(error),
};
}
function readEnvString(name: string, fallback: string): string {
const value = Deno.env.get(name)?.trim();
return value && value.length > 0 ? value : fallback;
}
function readEnvNumber(name: string, fallback: number): number {
const raw = Deno.env.get(name);
if (raw === undefined || raw.trim() === "") {
return fallback;
}
const parsed = Number(raw);
if (!Number.isFinite(parsed) || parsed < 0) {
throw new Error(`${name} must be a non-negative number, got '${raw}'`);
}
return parsed;
}
function nowMs(): number {
return performance.now();
}
async function commandOutput(command: string, args: string[]): Promise<string> {
const output = await new Deno.Command(command, {
args,
stdin: "null",
stdout: "piped",
stderr: "piped",
}).output();
const stdout = new TextDecoder().decode(output.stdout);
const stderr = new TextDecoder().decode(output.stderr);
if (!output.success) {
throw new Error(`${command} ${args.join(" ")} failed\nstdout: ${stdout}\nstderr: ${stderr}`);
}
return stdout.trim();
}
async function commandOk(command: string, args: string[]): Promise<void> {
await commandOutput(command, args);
}
async function applyNetemIfRequested(): Promise<NetemSummary> {
const enabled = readEnvString("BENCH_NETEM_ENABLED", "0") === "1";
const profile = readEnvString("NETEM_PROFILE", "home-wifi");
const iface = readEnvString("NETEM_INTERFACE", "eth0");
const delayMs = readEnvNumber("NETEM_DELAY_MS", 20);
const jitterMs = readEnvNumber("NETEM_JITTER_MS", 5);
const lossPercent = readEnvNumber("NETEM_LOSS_PERCENT", 0.1);
const bandwidthMbit = readEnvNumber("NETEM_BANDWIDTH_MBIT", 100);
const mtu = readEnvNumber("NETEM_MTU", 1500);
const summary: NetemSummary = {
enabled,
profile,
interface: iface,
delayMs,
jitterMs,
lossPercent,
bandwidthMbit,
mtu,
};
if (!enabled) {
return summary;
}
await commandOk("ip", ["link", "set", "dev", iface, "mtu", String(mtu)]);
await new Deno.Command("tc", { args: ["qdisc", "del", "dev", iface, "root"] }).output();
await commandOk("tc", [
"qdisc",
"add",
"dev",
iface,
"root",
"netem",
"delay",
`${delayMs}ms`,
`${jitterMs}ms`,
"loss",
`${lossPercent}%`,
"rate",
`${bandwidthMbit}mbit`,
]);
summary.tcQdisc = await commandOutput("tc", ["qdisc", "show", "dev", iface]);
summary.ipAddr = await commandOutput("ip", ["addr", "show", iface]);
summary.ipRoute = await commandOutput("ip", ["route"]);
return summary;
}
async function waitForFile(path: string, timeoutMs: number): Promise<void> {
const started = Date.now();
while (Date.now() - started < timeoutMs) {
try {
const stat = await Deno.stat(path);
if (stat.isFile) {
return;
}
} catch {
// wait
}
await new Promise((resolve) => setTimeout(resolve, 250));
}
throw new Error(`Timed out waiting for ${path}`);
}
async function readJsonFile<T>(path: string): Promise<T> {
return JSON.parse(await Deno.readTextFile(path)) as T;
}
function pickSampleFiles(entries: DatasetEntry[]): DatasetEntry[] {
const unique = new Map<string, DatasetEntry>();
for (const entry of [entries.find((e) => e.kind === "md"), entries.find((e) => e.kind === "bin"), entries.at(-1)]) {
if (entry) {
unique.set(entry.relativePath, entry);
}
}
return [...unique.values()];
}
async function readLatestP2PConnectionStats(path: string): Promise<P2PConnectionStats | undefined> {
try {
const lines = (await Deno.readTextFile(path))
.split(/\r?\n/)
.map((line) => line.trim())
.filter((line) => line.length > 0);
return lines.length === 0 ? undefined : (JSON.parse(lines.at(-1)!) as P2PConnectionStats);
} catch {
return undefined;
}
}
function buildCommonConfig() {
const runId = readEnvString("BENCH_SPLIT_RUN_ID", readEnvString("BENCH_ROOM_ID", "bench-split-run"));
const baseWorkRoot = readEnvString("BENCH_SPLIT_WORK_ROOT", "/p2p-work");
return {
runId,
workRoot: join(baseWorkRoot, runId),
resultRoot: readEnvString("BENCH_SPLIT_RESULT_ROOT", "/workspace/src/apps/cli/testdeno/bench-results"),
relay: readEnvString("BENCH_RELAY", "ws://nostr-relay:7777/"),
appId: readEnvString("BENCH_APP_ID", "self-hosted-livesync-cli-benchmark"),
roomId: readEnvString("BENCH_ROOM_ID", "bench-split-room"),
passphrase: readEnvString("BENCH_PASSPHRASE", "bench-split-passphrase"),
turnServers: readEnvString("BENCH_TURN_SERVERS", ""),
datasetDirName: readEnvString("BENCH_DATASET_DIR", "bench-dataset"),
datasetSeed: readEnvString("BENCH_SEED", "livesync-benchmark-seed"),
mdFileCount: Math.floor(readEnvNumber("BENCH_MD_FILE_COUNT", 20)),
mdMinSizeBytes: Math.floor(readEnvNumber("BENCH_MD_MIN_SIZE_BYTES", 512)),
mdMaxSizeBytes: Math.floor(readEnvNumber("BENCH_MD_MAX_SIZE_BYTES", 2048)),
binFileCount: Math.floor(readEnvNumber("BENCH_BIN_FILE_COUNT", 5)),
binSizeBytes: Math.floor(readEnvNumber("BENCH_BIN_SIZE_BYTES", 8192)),
peersTimeoutSeconds: readEnvNumber("BENCH_PEERS_TIMEOUT", 60),
syncTimeoutSeconds: readEnvNumber("BENCH_SYNC_TIMEOUT", 300),
nodeTimeoutMs: readEnvNumber("BENCH_SPLIT_NODE_TIMEOUT_MS", 360_000),
profile: readEnvString("BENCH_NETWORK_PROFILE", readEnvString("NETEM_PROFILE", "split-compose")),
};
}
async function prepareP2PSettings(
settingsPath: string,
peerName: string,
config: ReturnType<typeof buildCommonConfig>
) {
await initSettingsFile(settingsPath);
await applyP2pSettings(
settingsPath,
config.roomId,
config.passphrase,
config.appId,
config.relay,
"~.*",
config.turnServers
);
await applyP2pTestTweaks(settingsPath, peerName, config.passphrase);
}
async function runHost(): Promise<void> {
const config = buildCommonConfig();
const netem = await applyNetemIfRequested();
await Deno.mkdir(config.workRoot, { recursive: true });
await Deno.mkdir(config.resultRoot, { recursive: true });
const hostVault = join(config.workRoot, "vault-host");
const hostSettings = join(config.workRoot, "settings-host.json");
await Deno.mkdir(hostVault, { recursive: true });
await prepareP2PSettings(hostSettings, "p2p-split-host", config);
const seedFiles = await createDeterministicDataset({
rootDir: hostVault,
datasetDirName: config.datasetDirName,
seed: config.datasetSeed,
mdCount: config.mdFileCount,
mdMinSizeBytes: config.mdMinSizeBytes,
mdMaxSizeBytes: config.mdMaxSizeBytes,
binCount: config.binFileCount,
binSizeBytes: config.binSizeBytes,
});
await Deno.writeTextFile(
join(config.workRoot, "sample-files.json"),
JSON.stringify(pickSampleFiles(seedFiles.entries), null, 2)
);
const mirrorStart = nowMs();
await runCliOrFail(hostVault, "--settings", hostSettings, "mirror");
const mirrorElapsedMs = Number((nowMs() - mirrorStart).toFixed(1));
const hostReady: HostReady = {
generatedAt: new Date().toISOString(),
totalFiles: seedFiles.totalFiles,
totalBytes: seedFiles.totalBytes,
mdFileCount: seedFiles.mdCount,
binFileCount: seedFiles.binCount,
mirrorElapsedMs,
netem,
};
await Deno.writeTextFile(join(config.workRoot, "host-ready.json"), JSON.stringify(hostReady, null, 2));
const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host");
try {
await host.waitUntilContains("P2P host is running", 20_000);
await Deno.writeTextFile(
join(config.workRoot, "p2p-host-ready.json"),
JSON.stringify({ generatedAt: new Date().toISOString() })
);
await waitForFile(join(config.workRoot, "client-done.json"), config.nodeTimeoutMs);
} finally {
await host.stop();
}
}
async function runClient(): Promise<void> {
const config = buildCommonConfig();
const netem = await applyNetemIfRequested();
await Deno.mkdir(config.resultRoot, { recursive: true });
await waitForFile(join(config.workRoot, "host-ready.json"), config.nodeTimeoutMs);
await waitForFile(join(config.workRoot, "p2p-host-ready.json"), config.nodeTimeoutMs);
const clientVault = join(config.workRoot, "vault-client");
const clientSettings = join(config.workRoot, "settings-client.json");
const statsPath = join(config.workRoot, "p2p-connection-stats.jsonl");
await Deno.mkdir(clientVault, { recursive: true });
await prepareP2PSettings(clientSettings, "p2p-split-client", config);
const hostReady = await readJsonFile<HostReady>(join(config.workRoot, "host-ready.json"));
const timestamp = new Date().toISOString().replace(/[-:]/g, "").slice(0, 15);
const outputDir = join(config.resultRoot, `p2p-split-${config.profile}-${timestamp}`);
await Deno.mkdir(outputDir, { recursive: true });
const previousStatsPath = Deno.env.get("LIVESYNC_P2P_STATS_JSONL");
Deno.env.set("LIVESYNC_P2P_STATS_JSONL", statsPath);
let stage = "peer-discovery";
let peerDiscoveryCommandElapsedMs: number | undefined;
let syncElapsedMs: number | undefined;
try {
const peerDiscoveryCommandStart = nowMs();
const peer = await discoverPeer(clientVault, clientSettings, config.peersTimeoutSeconds);
peerDiscoveryCommandElapsedMs = Number((nowMs() - peerDiscoveryCommandStart).toFixed(1));
stage = "p2p-sync";
const syncStart = nowMs();
await runCliOrFail(
clientVault,
"--settings",
clientSettings,
"p2p-sync",
peer.id,
String(config.syncTimeoutSeconds)
);
syncElapsedMs = Number((nowMs() - syncStart).toFixed(1));
stage = "sample-verification";
const samples = await readJsonFile<DatasetEntry[]>(join(config.workRoot, "sample-files.json"));
for (const sample of samples) {
const pulledPath = join(config.workRoot, `pulled-${sample.relativePath.replaceAll("/", "_")}`);
await runCliOrFail(clientVault, "--settings", clientSettings, "pull", sample.relativePath, pulledPath);
await assertFilesEqual(
sample.absolutePath,
pulledPath,
`sample file mismatch after split sync: ${sample.relativePath}`
);
}
const p2pConnectionStats = await readLatestP2PConnectionStats(statsPath);
const result = {
ok: true,
generatedAt: new Date().toISOString(),
caseName: "p2p-split-compose",
mode: "p2p-split-compose-benchmark",
runId: config.runId,
simulationTier: Deno.env.get("BENCH_NETEM_ENABLED") === "1" ? "2" : "1",
networkProfile: config.profile,
networkModel:
Deno.env.get("BENCH_NETEM_ENABLED") === "1" ? "split-compose-egress-netem" : "split-compose-no-netem",
relay: config.relay,
turnServers: config.turnServers,
turnEnabled: config.turnServers.trim().length > 0,
p2pCandidatePathVerified: p2pConnectionStats?.candidatePathCollected === true,
p2pConnectionStats,
hostNetem: hostReady.netem,
clientNetem: netem,
totalFiles: hostReady.totalFiles,
totalBytes: hostReady.totalBytes,
mdFileCount: hostReady.mdFileCount,
binFileCount: hostReady.binFileCount,
mirrorElapsedMs: hostReady.mirrorElapsedMs,
peerDiscoveryTimeoutSeconds: config.peersTimeoutSeconds,
peerDiscoveryCommandElapsedMs,
syncElapsedMs,
throughputBytesPerSec: Number((hostReady.totalBytes / (syncElapsedMs / 1000)).toFixed(2)),
throughputMiBPerSec: Number((hostReady.totalBytes / (syncElapsedMs / 1000) / 1024 / 1024).toFixed(4)),
};
await Deno.writeTextFile(join(outputDir, "summary.json"), JSON.stringify(result, null, 2));
await Deno.writeTextFile(
join(config.workRoot, "client-done.json"),
JSON.stringify({ generatedAt: new Date().toISOString(), outputDir, ok: true })
);
console.log(JSON.stringify(result, null, 2));
} catch (error) {
const p2pConnectionStats = await readLatestP2PConnectionStats(statsPath);
const result = {
ok: false,
generatedAt: new Date().toISOString(),
caseName: "p2p-split-compose",
mode: "p2p-split-compose-benchmark",
runId: config.runId,
simulationTier: Deno.env.get("BENCH_NETEM_ENABLED") === "1" ? "2" : "1",
networkProfile: config.profile,
networkModel:
Deno.env.get("BENCH_NETEM_ENABLED") === "1" ? "split-compose-egress-netem" : "split-compose-no-netem",
relay: config.relay,
turnServers: config.turnServers,
turnEnabled: config.turnServers.trim().length > 0,
p2pCandidatePathVerified: p2pConnectionStats?.candidatePathCollected === true,
p2pConnectionStats,
hostNetem: hostReady.netem,
clientNetem: netem,
totalFiles: hostReady.totalFiles,
totalBytes: hostReady.totalBytes,
mdFileCount: hostReady.mdFileCount,
binFileCount: hostReady.binFileCount,
mirrorElapsedMs: hostReady.mirrorElapsedMs,
peerDiscoveryTimeoutSeconds: config.peersTimeoutSeconds,
peerDiscoveryCommandElapsedMs,
syncElapsedMs,
failure: {
stage,
...errorToRecord(error),
},
};
await Deno.writeTextFile(join(outputDir, "summary.json"), JSON.stringify(result, null, 2));
await Deno.writeTextFile(
join(config.workRoot, "client-done.json"),
JSON.stringify({ generatedAt: new Date().toISOString(), outputDir, ok: false })
);
console.log(JSON.stringify(result, null, 2));
throw error;
} finally {
if (previousStatsPath === undefined) {
Deno.env.delete("LIVESYNC_P2P_STATS_JSONL");
} else {
Deno.env.set("LIVESYNC_P2P_STATS_JSONL", previousStatsPath);
}
}
}
async function main(): Promise<void> {
const role = readEnvString("BENCH_P2P_SPLIT_ROLE", "") as Role;
if (role === "host") {
await runHost();
return;
}
if (role === "client") {
await runClient();
return;
}
throw new Error("BENCH_P2P_SPLIT_ROLE must be 'host' or 'client'");
}
if (import.meta.main) {
main().catch((error) => {
console.error("[Fatal Error]", error);
Deno.exit(1);
});
}

View File

@@ -1,15 +1,23 @@
import { TempDir } from "./helpers/temp.ts";
import { applyP2pSettings, applyP2pTestTweaks, initSettingsFile } from "./helpers/settings.ts";
import { startCliInBackground } from "./helpers/backgroundCli.ts";
import { discoverPeer, maybeStartLocalRelay, stopLocalRelayIfStarted } from "./helpers/p2p.ts";
import {
discoverPeer,
maybeStartCoturn,
maybeStartLocalRelay,
stopCoturnIfStarted,
stopLocalRelayIfStarted,
} from "./helpers/p2p.ts";
import { assertFilesEqual, runCliOrFail } from "./helpers/cli.ts";
import { createDeterministicDataset, type DatasetEntry } from "./helpers/dataset.ts";
type BenchmarkConfig = {
caseName: string;
relay: string;
appId: string;
roomId: string;
passphrase: string;
turnServers: string;
datasetDirName: string;
datasetSeed: string;
mdFileCount: number;
@@ -19,6 +27,42 @@ type BenchmarkConfig = {
binSizeBytes: number;
peersTimeoutSeconds: number;
syncTimeoutSeconds: number;
simulationTier: string;
networkProfile: string;
networkModel: string;
candidatePathVerification: string;
};
type P2PConnectionStats = {
generatedAt: string;
command: string;
peerId: string;
peerName: string;
candidatePathCollected: boolean;
selectedPath: string;
selectedPair?: {
id: string;
state: string;
currentRoundTripTime: number | "unknown";
totalRoundTripTime: number | "unknown";
requestsSent: number | "unknown";
responsesReceived: number | "unknown";
packetsDiscardedOnSend: number | "unknown";
bytesSent: number | "unknown";
bytesReceived: number | "unknown";
};
localCandidate?: {
id: string;
candidateType: string;
protocol: string;
relayProtocol: string;
};
remoteCandidate?: {
id: string;
candidateType: string;
protocol: string;
relayProtocol: string;
};
};
function readEnvString(name: string, fallback: string): string {
@@ -61,10 +105,12 @@ function formatBytes(value: number): string {
function buildConfig(): BenchmarkConfig {
return {
caseName: readEnvString("BENCH_CASE", "p2p-direct-local"),
relay: readEnvString("BENCH_RELAY", "ws://localhost:4000/"),
appId: readEnvString("BENCH_APP_ID", "self-hosted-livesync-cli-benchmark"),
roomId: readEnvString("BENCH_ROOM_ID", `bench-room-${Date.now()}`),
passphrase: readEnvString("BENCH_PASSPHRASE", `bench-${Date.now()}`),
turnServers: readEnvString("BENCH_TURN_SERVERS", ""),
datasetDirName: readEnvString("BENCH_DATASET_DIR", "bench-dataset"),
datasetSeed: readEnvString("BENCH_SEED", "livesync-benchmark-seed"),
mdFileCount: Math.floor(readEnvNumber("BENCH_MD_FILE_COUNT", 1500)),
@@ -74,6 +120,10 @@ function buildConfig(): BenchmarkConfig {
binSizeBytes: Math.floor(readEnvNumber("BENCH_BIN_SIZE_BYTES", 100 * 1024)),
peersTimeoutSeconds: readEnvNumber("BENCH_PEERS_TIMEOUT", 20),
syncTimeoutSeconds: readEnvNumber("BENCH_SYNC_TIMEOUT", 240),
simulationTier: readEnvString("BENCH_SIMULATION_TIER", "1"),
networkProfile: readEnvString("BENCH_NETWORK_PROFILE", "local-direct"),
networkModel: readEnvString("BENCH_NETWORK_MODEL", "local-runner-webrtc"),
candidatePathVerification: readEnvString("BENCH_P2P_CANDIDATE_PATH_VERIFICATION", "not-collected"),
};
}
@@ -102,115 +152,182 @@ function pickSampleFiles(entries: DatasetEntry[]): DatasetEntry[] {
return [...unique.values()];
}
async function readLatestP2PConnectionStats(statsPath: string): Promise<P2PConnectionStats | undefined> {
try {
const text = await Deno.readTextFile(statsPath);
const lines = text
.split(/\r?\n/)
.map((line) => line.trim())
.filter((line) => line.length > 0);
if (lines.length === 0) {
return undefined;
}
return JSON.parse(lines[lines.length - 1]) as P2PConnectionStats;
} catch {
return undefined;
}
}
async function main(): Promise<void> {
const config = buildConfig();
const resultPath = readOptionalResultPath();
const relayStarted = await maybeStartLocalRelay(config.relay);
const coturnStarted = await maybeStartCoturn(config.turnServers);
await using workDir = await TempDir.create("livesync-cli-p2p-bench");
const hostVault = workDir.join("vault-host");
const clientVault = workDir.join("vault-client");
const hostSettings = workDir.join("settings-host.json");
const clientSettings = workDir.join("settings-client.json");
const p2pStatsPath = workDir.join("p2p-connection-stats.jsonl");
const previousStatsPath = Deno.env.get("LIVESYNC_P2P_STATS_JSONL");
Deno.env.set("LIVESYNC_P2P_STATS_JSONL", p2pStatsPath);
await Promise.all([
Deno.mkdir(hostVault, { recursive: true }),
Deno.mkdir(clientVault, { recursive: true }),
initSettingsFile(hostSettings),
initSettingsFile(clientSettings),
]);
await Promise.all([
applyP2pSettings(hostSettings, config.roomId, config.passphrase, config.appId, config.relay, "~.*"),
applyP2pSettings(clientSettings, config.roomId, config.passphrase, config.appId, config.relay, "~.*"),
]);
await Promise.all([
applyP2pTestTweaks(hostSettings, "p2p-bench-host", config.passphrase),
applyP2pTestTweaks(clientSettings, "p2p-bench-client", config.passphrase),
]);
const seedFiles = await createDeterministicDataset({
rootDir: hostVault,
datasetDirName: config.datasetDirName,
seed: config.datasetSeed,
mdCount: config.mdFileCount,
mdMinSizeBytes: config.mdMinSizeBytes,
mdMaxSizeBytes: config.mdMaxSizeBytes,
binCount: config.binFileCount,
binSizeBytes: config.binSizeBytes,
});
const mirrorStart = nowMs();
await runCliOrFail(hostVault, "--settings", hostSettings, "mirror");
const mirrorElapsed = nowMs() - mirrorStart;
const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host");
try {
const hostReadyStart = nowMs();
await host.waitUntilContains("P2P host is running", 20000);
const hostReadyElapsed = nowMs() - hostReadyStart;
await Promise.all([
Deno.mkdir(hostVault, { recursive: true }),
Deno.mkdir(clientVault, { recursive: true }),
initSettingsFile(hostSettings),
initSettingsFile(clientSettings),
]);
const peerDiscoveryStart = nowMs();
const peer = await discoverPeer(clientVault, clientSettings, config.peersTimeoutSeconds);
const peerDiscoveryElapsed = nowMs() - peerDiscoveryStart;
await Promise.all([
applyP2pSettings(
hostSettings,
config.roomId,
config.passphrase,
config.appId,
config.relay,
"~.*",
config.turnServers
),
applyP2pSettings(
clientSettings,
config.roomId,
config.passphrase,
config.appId,
config.relay,
"~.*",
config.turnServers
),
]);
const syncStart = nowMs();
await runCliOrFail(
clientVault,
"--settings",
clientSettings,
"p2p-sync",
peer.id,
String(config.syncTimeoutSeconds)
);
const syncElapsed = nowMs() - syncStart;
await Promise.all([
applyP2pTestTweaks(hostSettings, "p2p-bench-host", config.passphrase),
applyP2pTestTweaks(clientSettings, "p2p-bench-client", config.passphrase),
]);
const sampleFiles = pickSampleFiles(seedFiles.entries);
for (const sample of sampleFiles) {
const pulledPath = workDir.join(`pulled-${sample.relativePath.replaceAll("/", "_")}`);
await runCliOrFail(clientVault, "--settings", clientSettings, "pull", sample.relativePath, pulledPath);
await assertFilesEqual(
sample.absolutePath,
pulledPath,
`sample file mismatch after sync: ${sample.relativePath}`
);
}
const result = {
mode: "p2p-cli-benchmark",
relay: config.relay,
appId: config.appId,
roomId: config.roomId,
datasetSeed: config.datasetSeed,
const seedFiles = await createDeterministicDataset({
rootDir: hostVault,
datasetDirName: config.datasetDirName,
peerId: peer.id,
peerName: peer.name,
totalFiles: seedFiles.totalFiles,
totalBytes: seedFiles.totalBytes,
mdFileCount: seedFiles.mdCount,
binFileCount: seedFiles.binCount,
mirrorElapsedMs: Number(mirrorElapsed.toFixed(1)),
hostReadyElapsedMs: Number(hostReadyElapsed.toFixed(1)),
peerDiscoveryElapsedMs: Number(peerDiscoveryElapsed.toFixed(1)),
syncElapsedMs: Number(syncElapsed.toFixed(1)),
throughputBytesPerSec: Number((seedFiles.totalBytes / (syncElapsed / 1000)).toFixed(2)),
throughputMiBPerSec: Number((seedFiles.totalBytes / (syncElapsed / 1000) / 1024 / 1024).toFixed(4)),
};
seed: config.datasetSeed,
mdCount: config.mdFileCount,
mdMinSizeBytes: config.mdMinSizeBytes,
mdMaxSizeBytes: config.mdMaxSizeBytes,
binCount: config.binFileCount,
binSizeBytes: config.binSizeBytes,
});
if (resultPath) {
await Deno.writeTextFile(resultPath, JSON.stringify(result, null, 2));
const mirrorStart = nowMs();
await runCliOrFail(hostVault, "--settings", hostSettings, "mirror");
const mirrorElapsed = nowMs() - mirrorStart;
const host = startCliInBackground(hostVault, "--settings", hostSettings, "p2p-host");
try {
const hostReadyStart = nowMs();
await host.waitUntilContains("P2P host is running", 20000);
const hostReadyElapsed = nowMs() - hostReadyStart;
const peerDiscoveryCommandStart = nowMs();
const peer = await discoverPeer(clientVault, clientSettings, config.peersTimeoutSeconds);
const peerDiscoveryCommandElapsed = nowMs() - peerDiscoveryCommandStart;
const syncStart = nowMs();
await runCliOrFail(
clientVault,
"--settings",
clientSettings,
"p2p-sync",
peer.id,
String(config.syncTimeoutSeconds)
);
const syncElapsed = nowMs() - syncStart;
const sampleFiles = pickSampleFiles(seedFiles.entries);
for (const sample of sampleFiles) {
const pulledPath = workDir.join(`pulled-${sample.relativePath.replaceAll("/", "_")}`);
await runCliOrFail(clientVault, "--settings", clientSettings, "pull", sample.relativePath, pulledPath);
await assertFilesEqual(
sample.absolutePath,
pulledPath,
`sample file mismatch after sync: ${sample.relativePath}`
);
}
const p2pConnectionStats = await readLatestP2PConnectionStats(p2pStatsPath);
const result = {
caseName: config.caseName,
mode: "p2p-cli-benchmark",
relay: config.relay,
turnServers: config.turnServers,
turnEnabled: config.turnServers.trim().length > 0,
simulationTier: config.simulationTier,
networkProfile: config.networkProfile,
networkModel: config.networkModel,
p2pCandidatePathVerified: p2pConnectionStats?.candidatePathCollected === true,
p2pCandidatePathVerification: p2pConnectionStats?.candidatePathCollected
? "selected ICE candidate pair collected from RTCPeerConnection.getStats"
: config.candidatePathVerification,
p2pCandidatePathNote: p2pConnectionStats?.candidatePathCollected
? "The selected ICE candidate pair was collected by the CLI benchmark. Interpret the path from the candidate types; do not infer TURN use from configuration alone."
: config.turnServers.trim().length > 0
? "TURN is configured, so the selected WebRTC path may be direct, server-reflexive, or relayed. The selected ICE candidate pair was not exported by this run."
: "TURN is disabled, so a TURN-relayed path is not expected. The selected ICE candidate pair was not exported by this run.",
p2pConnectionStats,
appId: config.appId,
roomId: config.roomId,
datasetSeed: config.datasetSeed,
datasetDirName: config.datasetDirName,
peerId: peer.id,
peerName: peer.name,
totalFiles: seedFiles.totalFiles,
totalBytes: seedFiles.totalBytes,
mdFileCount: seedFiles.mdCount,
binFileCount: seedFiles.binCount,
mirrorElapsedMs: Number(mirrorElapsed.toFixed(1)),
hostReadyElapsedMs: Number(hostReadyElapsed.toFixed(1)),
peerDiscoveryTimeoutSeconds: config.peersTimeoutSeconds,
peerDiscoveryCommandElapsedMs: Number(peerDiscoveryCommandElapsed.toFixed(1)),
peerDiscoveryNote:
"p2p-peers waits for the requested timeout before printing discovered peers, so this is command duration, not first-peer latency.",
syncElapsedMs: Number(syncElapsed.toFixed(1)),
throughputBytesPerSec: Number((seedFiles.totalBytes / (syncElapsed / 1000)).toFixed(2)),
throughputMiBPerSec: Number((seedFiles.totalBytes / (syncElapsed / 1000) / 1024 / 1024).toFixed(4)),
};
if (resultPath) {
await Deno.writeTextFile(resultPath, JSON.stringify(result, null, 2));
}
console.log(JSON.stringify(result, null, 2));
console.error(
`[Benchmark] mirrored ${seedFiles.totalFiles} files (${formatBytes(
seedFiles.totalBytes
)}) in ${formatMs(mirrorElapsed)}, ` +
`synced in ${formatMs(syncElapsed)} ` +
`(${result.throughputBytesPerSec} B/s, ${result.throughputMiBPerSec} MiB/s)`
);
} finally {
await host.stop();
}
console.log(JSON.stringify(result, null, 2));
console.error(
`[Benchmark] mirrored ${seedFiles.totalFiles} files (${formatBytes(seedFiles.totalBytes)}) in ${formatMs(mirrorElapsed)}, ` +
`synced in ${formatMs(syncElapsed)} ` +
`(${result.throughputBytesPerSec} B/s, ${result.throughputMiBPerSec} MiB/s)`
);
} finally {
await host.stop();
if (previousStatsPath === undefined) {
Deno.env.delete("LIVESYNC_P2P_STATS_JSONL");
} else {
Deno.env.set("LIVESYNC_P2P_STATS_JSONL", previousStatsPath);
}
await stopCoturnIfStarted(coturnStarted);
await stopLocalRelayIfStarted(relayStarted);
}
}

View File

@@ -17,6 +17,9 @@
"test:p2p-upload-download": "deno test --env-file=.test.env -A --no-check test-p2p-upload-download-repro.ts",
"bench:p2p": "deno run --env-file=.test.env -A --no-check bench-p2p.ts",
"bench:couchdb": "deno run --env-file=.test.env -A --no-check bench-couchdb.ts",
"bench:cases": "deno run --env-file=.test.env -A --no-check bench-network-cases.ts",
"bench:latency-sweep": "deno run --env-file=.test.env -A --no-check bench-latency-sweep.ts",
"bench:p2p-split-node": "deno run --env-file=.test.env -A --no-check bench-p2p-split-node.ts",
"bench:item1": "bash ./bench-run-item1.sh",
"bench:item1:full": "BENCH_MD_FILE_COUNT=1500 BENCH_MD_MIN_SIZE_BYTES=1024 BENCH_MD_MAX_SIZE_BYTES=20480 BENCH_BIN_FILE_COUNT=500 BENCH_BIN_SIZE_BYTES=102400 BENCH_COUCHDB_RTT_MS=50 bash ./bench-run-item1.sh",
"test:e2e-couchdb": "deno test --env-file=.test.env -A --no-check test-e2e-two-vaults-couchdb.ts",

View File

@@ -9,7 +9,7 @@ function sleep(ms: number): Promise<void> {
}
async function connectWithTimeout(hostname: string, port: number, timeoutMs: number): Promise<void> {
let timer: number | undefined;
let timer: ReturnType<typeof setTimeout> | undefined;
try {
const connPromise = Deno.connect({ hostname, port });
const timeoutPromise = new Promise<never>((_, reject) => {

View File

@@ -76,8 +76,10 @@ export async function discoverPeer(
}
export async function maybeStartLocalRelay(relay: string): Promise<boolean> {
if (!isLocalP2pRelay(relay)) return false;
await startP2pRelay();
const shouldStart = isLocalP2pRelay(relay);
if (shouldStart) {
await startP2pRelay();
}
const endpoint = parseRelayEndpoint(relay);
await waitForPort(endpoint.hostname, endpoint.port, {
timeoutMs: Number(Deno.env.get("LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS") ?? "15000"),
@@ -86,8 +88,10 @@ export async function maybeStartLocalRelay(relay: string): Promise<boolean> {
});
// Docker proxy accepts TCP connections instantly before the container's internal process is fully ready.
// Wait an additional few seconds to ensure strfry is actually accepting WebSockets.
await sleep(3000);
return true;
if (shouldStart) {
await sleep(3000);
}
return shouldStart;
}
export async function stopLocalRelayIfStarted(started: boolean): Promise<void> {

View File

@@ -1,7 +1,7 @@
{
"name": "livesync-webapp",
"private": true,
"version": "0.25.79-webapp",
"version": "0.25.80-webapp",
"type": "module",
"description": "Browser-based Self-hosted LiveSync using FileSystem API",
"scripts": {

View File

@@ -1,7 +1,7 @@
{
"name": "webpeer",
"private": true,
"version": "0.25.79-webpeer",
"version": "0.25.80-webpeer",
"type": "module",
"scripts": {
"dev": "vite",

View File

@@ -51,6 +51,7 @@ import { EVENT_SETTING_SAVED, eventHub } from "@/common/events.ts";
import { Semaphore } from "octagonal-wheels/concurrency/semaphore";
import type { LiveSyncCore } from "@/main.ts";
import { tryGetFilePath } from "@lib/common/utils.doc.ts";
import { configureHiddenFileSyncMode } from "./configureHiddenFileSyncMode.ts";
type SyncDirection = "push" | "pull" | "safe" | "pullForce" | "pushForce";
declare global {
@@ -1832,43 +1833,35 @@ ${messageFetch}${messageOverwrite}${messageMerge}
}
async configureHiddenFileSync(mode: keyof OPTIONAL_SYNC_FEATURES) {
if (
mode != "FETCH" &&
mode != "OVERWRITE" &&
mode != "MERGE" &&
mode != "DISABLE" &&
mode != "DISABLE_HIDDEN"
) {
return;
}
if (mode == "DISABLE" || mode == "DISABLE_HIDDEN") {
// await this.core.$allSuspendExtraSync();
await this.core.services.setting.applyPartial(
{
syncInternalFiles: false,
},
true
);
// this.core.settings.syncInternalFiles = false;
// await this.core.saveSettings();
return;
}
this._log("Gathering files for enabling Hidden File Sync", LOG_LEVEL_NOTICE);
if (mode == "FETCH") {
await this.initialiseInternalFileSync("pullForce", true);
} else if (mode == "OVERWRITE") {
await this.initialiseInternalFileSync("pushForce", true);
} else if (mode == "MERGE") {
await this.initialiseInternalFileSync("safe", true);
}
await this.core.services.setting.applyPartial(
{
useAdvancedMode: true,
syncInternalFiles: true,
const result = await configureHiddenFileSyncMode(mode, {
disable: async () => {
// await this.core.$allSuspendExtraSync();
await this.core.services.setting.applyPartial(
{
syncInternalFiles: false,
},
true
);
// this.core.settings.syncInternalFiles = false;
// await this.core.saveSettings();
},
true
);
enable: async () => {
this._log("Gathering files for enabling Hidden File Sync", LOG_LEVEL_NOTICE);
await this.core.services.setting.applyPartial(
{
useAdvancedMode: true,
syncInternalFiles: true,
},
true
);
},
initialise: async (direction) => {
await this.initialiseInternalFileSync(direction, true);
},
});
if (result == "ignored" || result == "disabled") {
return;
}
// this.plugin.settings.useAdvancedMode = true;
// this.plugin.settings.syncInternalFiles = true;

View File

@@ -0,0 +1,45 @@
type HiddenFileSyncMode = "FETCH" | "OVERWRITE" | "MERGE" | "DISABLE" | "DISABLE_HIDDEN";
type HiddenFileSyncDirection = "pullForce" | "pushForce" | "safe";
type ConfigureHiddenFileSyncHandlers = {
disable: () => Promise<void>;
enable: () => Promise<void>;
initialise: (direction: HiddenFileSyncDirection) => Promise<void>;
};
export type ConfigureHiddenFileSyncResult = "ignored" | "disabled" | "enabled";
function getInitialiseDirection(mode: keyof OPTIONAL_SYNC_FEATURES): HiddenFileSyncDirection | false {
if (mode == "FETCH") return "pullForce";
if (mode == "OVERWRITE") return "pushForce";
if (mode == "MERGE") return "safe";
return false;
}
function isDisableMode(mode: keyof OPTIONAL_SYNC_FEATURES): boolean {
return mode == "DISABLE" || mode == "DISABLE_HIDDEN";
}
function isHiddenFileSyncMode(mode: keyof OPTIONAL_SYNC_FEATURES): mode is HiddenFileSyncMode {
return mode == "FETCH" || mode == "OVERWRITE" || mode == "MERGE" || isDisableMode(mode);
}
export async function configureHiddenFileSyncMode(
mode: keyof OPTIONAL_SYNC_FEATURES,
handlers: ConfigureHiddenFileSyncHandlers
): Promise<ConfigureHiddenFileSyncResult> {
if (!isHiddenFileSyncMode(mode)) {
return "ignored";
}
if (isDisableMode(mode)) {
await handlers.disable();
return "disabled";
}
const direction = getInitialiseDirection(mode);
if (direction === false) {
return "ignored";
}
await handlers.enable();
await handlers.initialise(direction);
return "enabled";
}

View File

@@ -0,0 +1,47 @@
import { describe, expect, it, vi } from "vitest";
import { configureHiddenFileSyncMode } from "./configureHiddenFileSyncMode.ts";
describe("configureHiddenFileSyncMode", () => {
it.each([
["FETCH", "pullForce"],
["OVERWRITE", "pushForce"],
["MERGE", "safe"],
] as const)("enables hidden file sync before initialising %s", async (mode, direction) => {
const calls: string[] = [];
const result = await configureHiddenFileSyncMode(mode, {
disable: vi.fn(async () => {
calls.push("disable");
}),
enable: vi.fn(async () => {
calls.push("enable");
}),
initialise: vi.fn(async (actualDirection) => {
calls.push(`init:${actualDirection}`);
}),
});
expect(result).toBe("enabled");
expect(calls).toEqual(["enable", `init:${direction}`]);
});
it.each(["DISABLE", "DISABLE_HIDDEN"] as const)("disables hidden file sync immediately for %s", async (mode) => {
const calls: string[] = [];
const result = await configureHiddenFileSyncMode(mode, {
disable: vi.fn(async () => {
calls.push("disable");
}),
enable: vi.fn(async () => {
calls.push("enable");
}),
initialise: vi.fn(async (direction) => {
calls.push(`init:${direction}`);
}),
});
expect(result).toBe("disabled");
expect(calls).toEqual(["disable"]);
});
});

Submodule src/lib updated: c7c5fe21be...a0efb7274e

View File

@@ -8,13 +8,7 @@ import {
type ValueComponent,
} from "@/deps.ts";
import { unique } from "octagonal-wheels/collection";
import {
LEVEL_ADVANCED,
LEVEL_POWER_USER,
statusDisplay,
type ConfigurationItem,
type SettingDefinition,
} from "@lib/common/types.ts";
import { LEVEL_ADVANCED, LEVEL_POWER_USER, statusDisplay, type ConfigurationItem } from "@lib/common/types.ts";
import { type ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
import {
type AllSettingItemKey,
@@ -24,21 +18,9 @@ import {
type AllNumericItemKey,
type AllBooleanItemKey,
} from "./settingConstants.ts";
import { $msg, $t } from "@lib/common/i18n.ts";
import { $msg } from "@lib/common/i18n.ts";
import { wrapMemo, type AutoWireOption, type OnUpdateResult } from "./SettingPane.ts";
function configurationFromDefinition(definition: SettingDefinition<AllSettingItemKey>): ConfigurationItem {
return {
name: $t(definition.labelKey),
desc: definition.descriptionKey ? $t(definition.descriptionKey) : undefined,
placeHolder: definition.placeholder,
status: definition.status,
obsolete: definition.obsolete,
level: definition.level,
isHidden: definition.secret,
};
}
export class LiveSyncSetting extends Setting {
autoWiredComponent?: TextComponent | ToggleComponent | DropdownComponent | ButtonComponent | TextAreaComponent;
applyButtonComponent?: ButtonComponent;
@@ -74,7 +56,7 @@ export class LiveSyncSetting extends Setting {
return this;
}
autoWireSetting(key: AllSettingItemKey, opt?: AutoWireOption) {
const conf = opt?.settingDefinition ? configurationFromDefinition(opt.settingDefinition) : getConfig(key);
const conf = getConfig(key);
if (!conf) {
// throw new Error($msg("liveSyncSetting.errorNoSuchSettingItem", { key }));
return;

View File

@@ -1,76 +0,0 @@
import {
getExplicitSettingCommitGroup,
getSettingDefinition,
type AllBooleanItemKey,
type AllNumericItemKey,
type AllSettingItemKey,
type AllStringItemKey,
} from "./settingConstants.ts";
import type { LiveSyncSetting } from "./LiveSyncSetting.ts";
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
import type { AutoWireOption } from "./SettingPane.ts";
export type ObsidianSettingRenderOption<TOption extends string = string> = AutoWireOption & {
options?: Record<TOption, string>;
clampMin?: number;
clampMax?: number;
acceptZero?: boolean;
renderInternal?: boolean;
};
export function addObsidianApplyButton(setting: LiveSyncSetting, group: string, text?: string): LiveSyncSetting {
const commitGroup = getExplicitSettingCommitGroup(group);
if (!commitGroup) {
throw new Error(`No explicit setting commit group found for '${group}'`);
}
return setting.addApplyButton(commitGroup.applyKeys, text);
}
export function renderObsidianApplyButton(containerEl: HTMLElement, group: string, text?: string): LiveSyncSetting {
return addObsidianApplyButton(new Setting(containerEl), group, text);
}
export function renderObsidianSetting<TOption extends string = string>(
containerEl: HTMLElement,
key: AllSettingItemKey,
opt: ObsidianSettingRenderOption<TOption> = {}
): LiveSyncSetting {
const definition = getSettingDefinition(key);
if (!definition) {
throw new Error(`No setting definition found for '${key}'`);
}
if (definition.internal && !opt.renderInternal) {
throw new Error(`Setting '${key}' is internal and cannot be rendered automatically`);
}
if (definition.render === "custom" || definition.kind === "custom") {
throw new Error(`Setting '${key}' requires a custom renderer`);
}
const isExplicitCommit = definition.commit?.mode === "explicit";
const holdValue = opt.holdValue ?? isExplicitCommit;
const setting = new Setting(containerEl);
const wireOption = {
...opt,
holdValue,
settingDefinition: definition,
};
if (opt.options) {
return setting.autoWireDropDown(key as AllStringItemKey, { ...wireOption, options: opt.options });
}
switch (definition.kind) {
case "boolean":
return setting.autoWireToggle(key as AllBooleanItemKey, wireOption);
case "number":
return setting.autoWireNumeric(key as AllNumericItemKey, wireOption);
case "password":
return setting.autoWireText(key as AllStringItemKey, { ...wireOption, isPassword: true });
case "textarea":
return setting.autoWireTextArea(key as AllStringItemKey, wireOption);
case "select":
throw new Error(`Setting '${key}' requires select options`);
case "text":
return setting.autoWireText(key as AllStringItemKey, wireOption);
}
}

View File

@@ -1,37 +1,41 @@
import { ChunkAlgorithmNames } from "@lib/common/types.ts";
import { renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
import type { PageFunctions } from "./SettingPane.ts";
export function paneAdvanced(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement, { addPanel }: PageFunctions): void {
void addPanel(paneEl, "Memory cache").then((paneEl) => {
renderObsidianSetting(paneEl, "hashCacheMaxCount", { clampMin: 10 });
// renderObsidianSetting(paneEl, "hashCacheMaxAmount", { clampMin: 1 });
new Setting(paneEl).autoWireNumeric("hashCacheMaxCount", { clampMin: 10 });
// new Setting(paneEl).autoWireNumeric("hashCacheMaxAmount", { clampMin: 1 });
});
void addPanel(paneEl, "Local Database Tweak").then((paneEl) => {
paneEl.addClass("wizardHidden");
const items = ChunkAlgorithmNames;
renderObsidianSetting(paneEl, "chunkSplitterVersion", {
new Setting(paneEl).autoWireDropDown("chunkSplitterVersion", {
options: items,
});
renderObsidianSetting(paneEl, "customChunkSize", { clampMin: 0, acceptZero: true });
new Setting(paneEl).autoWireNumeric("customChunkSize", { clampMin: 0, acceptZero: true });
});
void addPanel(paneEl, "Transfer Tweak").then((paneEl) => {
renderObsidianSetting(paneEl, "readChunksOnline", { onUpdate: this.onlyOnCouchDB }).setClass("wizardHidden");
renderObsidianSetting(paneEl, "useOnlyLocalChunk", { onUpdate: this.onlyOnCouchDB }).setClass("wizardHidden");
new Setting(paneEl)
.setClass("wizardHidden")
.autoWireToggle("readChunksOnline", { onUpdate: this.onlyOnCouchDB });
new Setting(paneEl)
.setClass("wizardHidden")
.autoWireToggle("useOnlyLocalChunk", { onUpdate: this.onlyOnCouchDB });
renderObsidianSetting(paneEl, "concurrencyOfReadChunksOnline", {
new Setting(paneEl).setClass("wizardHidden").autoWireNumeric("concurrencyOfReadChunksOnline", {
clampMin: 10,
onUpdate: this.onlyOnCouchDB,
}).setClass("wizardHidden");
});
renderObsidianSetting(paneEl, "minimumIntervalOfReadChunksOnline", {
new Setting(paneEl).setClass("wizardHidden").autoWireNumeric("minimumIntervalOfReadChunksOnline", {
clampMin: 10,
onUpdate: this.onlyOnCouchDB,
}).setClass("wizardHidden");
renderObsidianSetting(paneEl, "autoAcceptCompatibleTweak").setClass("wizardHidden");
});
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("autoAcceptCompatibleTweak");
// new Setting(paneEl)
// .setClass("wizardHidden")
// .autoWireToggle("sendChunksBulk", { onUpdate: onlyOnCouchDB })

View File

@@ -1,7 +1,6 @@
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
import { EVENT_REQUEST_OPEN_PLUGIN_SYNC_DIALOG, eventHub } from "@/common/events.ts";
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
import { renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
import type { PageFunctions } from "./SettingPane.ts";
import { enableOnly, visibleOnly } from "./SettingPane.ts";
export function paneCustomisationSync(
@@ -36,27 +35,27 @@ export function paneCustomisationSync(
visibleOnly(() => this.isConfiguredAs("usePluginSync", true))
);
renderObsidianSetting(paneEl, "deviceAndVaultName", {
new Setting(paneEl).autoWireText("deviceAndVaultName", {
placeHolder: "desktop",
onUpdate: enableOnlyOnPluginSyncIsNotEnabled,
});
renderObsidianSetting(paneEl, "usePluginSyncV2");
new Setting(paneEl).autoWireToggle("usePluginSyncV2");
renderObsidianSetting(paneEl, "usePluginSync", {
new Setting(paneEl).autoWireToggle("usePluginSync", {
onUpdate: enableOnly(() => !this.isConfiguredAs("deviceAndVaultName", "")),
});
renderObsidianSetting(paneEl, "autoSweepPlugins", {
new Setting(paneEl).autoWireToggle("autoSweepPlugins", {
onUpdate: visibleOnlyOnPluginSyncEnabled,
});
renderObsidianSetting(paneEl, "autoSweepPluginsPeriodic", {
new Setting(paneEl).autoWireToggle("autoSweepPluginsPeriodic", {
onUpdate: visibleOnly(
() => this.isConfiguredAs("usePluginSync", true) && this.isConfiguredAs("autoSweepPlugins", true)
),
});
renderObsidianSetting(paneEl, "notifyPluginOrSettingUpdated", {
new Setting(paneEl).autoWireToggle("notifyPluginOrSettingUpdated", {
onUpdate: visibleOnlyOnPluginSyncEnabled,
});

View File

@@ -1,7 +1,6 @@
import { $msg, $t } from "@lib/common/i18n.ts";
import { SUPPORTED_I18N_LANGS, type I18N_LANGS } from "@lib/common/rosetta.ts";
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
import { renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
import type { PageFunctions } from "./SettingPane.ts";
import { visibleOnly } from "./SettingPane.ts";
@@ -17,20 +16,20 @@ export function paneGeneral(
// ["", $msg("obsidianLiveSyncSettingTab.defaultLanguage")],
...SUPPORTED_I18N_LANGS.map((e) => [e, $t(`lang-${e}`)]),
]) as Record<I18N_LANGS, string>;
renderObsidianSetting(paneEl, "displayLanguage", {
new Setting(paneEl).autoWireDropDown("displayLanguage", {
options: languages,
});
this.addOnSaved("displayLanguage", () => this.display());
renderObsidianSetting(paneEl, "showStatusOnEditor");
new Setting(paneEl).autoWireToggle("showStatusOnEditor");
this.addOnSaved("showStatusOnEditor", () => {
eventHub.emitEvent(EVENT_ON_UNRESOLVED_ERROR);
});
renderObsidianSetting(paneEl, "showOnlyIconsOnEditor", {
new Setting(paneEl).autoWireToggle("showOnlyIconsOnEditor", {
onUpdate: visibleOnly(() => this.isConfiguredAs("showStatusOnEditor", true)),
});
renderObsidianSetting(paneEl, "showStatusOnStatusbar");
renderObsidianSetting(paneEl, "hideFileWarningNotice");
renderObsidianSetting(paneEl, "networkWarningStyle", {
new Setting(paneEl).autoWireToggle("showStatusOnStatusbar");
new Setting(paneEl).autoWireToggle("hideFileWarningNotice");
new Setting(paneEl).autoWireDropDown("networkWarningStyle", {
options: {
[NetworkWarningStyles.BANNER]: "Show full banner",
[NetworkWarningStyles.ICON]: "Show icon only",
@@ -44,9 +43,9 @@ export function paneGeneral(
void addPanel(paneEl, $msg("obsidianLiveSyncSettingTab.titleLogging")).then((paneEl) => {
paneEl.addClass("wizardHidden");
renderObsidianSetting(paneEl, "lessInformationInLog");
new Setting(paneEl).autoWireToggle("lessInformationInLog");
renderObsidianSetting(paneEl, "showVerboseLog", {
new Setting(paneEl).autoWireToggle("showVerboseLog", {
onUpdate: visibleOnly(() => this.isConfiguredAs("lessInformationInLog", false)),
});
});

View File

@@ -25,7 +25,6 @@ import { ICHeader, ICXHeader, PSCHeader } from "@/common/types.ts";
import { HiddenFileSync } from "@/features/HiddenFileSync/CmdHiddenFileSync.ts";
import { EVENT_REQUEST_SHOW_HISTORY } from "@/common/obsidianEvents.ts";
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
import { renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
import type { PageFunctions } from "./SettingPane.ts";
import { isNotFoundError } from "@lib/common/utils.doc.ts";
export function paneHatch(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement, { addPanel }: PageFunctions): void {
@@ -88,14 +87,14 @@ export function paneHatch(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement,
eventHub.emitEvent(EVENT_REQUEST_CHECK_REMOTE_SIZE);
})
);
renderObsidianSetting(paneEl, "writeLogToTheFile");
new Setting(paneEl).autoWireToggle("writeLogToTheFile");
});
void addPanel(paneEl, "Scram Switches").then((paneEl) => {
renderObsidianSetting(paneEl, "suspendFileWatching");
new Setting(paneEl).autoWireToggle("suspendFileWatching");
this.addOnSaved("suspendFileWatching", () => this.services.appLifecycle.askRestart());
renderObsidianSetting(paneEl, "suspendParseReplicationResult");
new Setting(paneEl).autoWireToggle("suspendParseReplicationResult");
this.addOnSaved("suspendParseReplicationResult", () => this.services.appLifecycle.askRestart());
});

View File

@@ -7,7 +7,6 @@ import {
} from "@lib/common/types.ts";
import { Logger } from "@lib/common/logger.ts";
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
import { addObsidianApplyButton, renderObsidianApplyButton, renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
import type { PageFunctions } from "./SettingPane.ts";
import { visibleOnly } from "./SettingPane.ts";
@@ -17,17 +16,17 @@ import { migrateDatabases } from "./settingUtils.ts";
export function panePatches(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement, { addPanel }: PageFunctions): void {
void addPanel(paneEl, "Compatibility (Metadata)").then((paneEl) => {
renderObsidianSetting(paneEl, "deleteMetadataOfDeletedFiles").setClass("wizardHidden");
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("deleteMetadataOfDeletedFiles");
renderObsidianSetting(paneEl, "automaticallyDeleteMetadataOfDeletedFiles", {
new Setting(paneEl).setClass("wizardHidden").autoWireNumeric("automaticallyDeleteMetadataOfDeletedFiles", {
onUpdate: visibleOnly(() => this.isConfiguredAs("deleteMetadataOfDeletedFiles", true)),
}).setClass("wizardHidden");
});
});
void addPanel(paneEl, "Compatibility (Conflict Behaviour)").then((paneEl) => {
paneEl.addClass("wizardHidden");
renderObsidianSetting(paneEl, "disableMarkdownAutoMerge").setClass("wizardHidden");
renderObsidianSetting(paneEl, "writeDocumentsIfConflicted").setClass("wizardHidden");
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("disableMarkdownAutoMerge");
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("writeDocumentsIfConflicted");
});
void addPanel(paneEl, "Compatibility (Database structure)").then((paneEl) => {
@@ -114,18 +113,18 @@ export function panePatches(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElemen
});
}
}
renderObsidianSetting(paneEl, "handleFilenameCaseSensitive", { holdValue: true }).setClass("wizardHidden");
new Setting(paneEl).autoWireToggle("handleFilenameCaseSensitive", { holdValue: true }).setClass("wizardHidden");
});
void addPanel(paneEl, "Compatibility (Internal API Usage)").then((paneEl) => {
renderObsidianSetting(paneEl, "watchInternalFileChanges", { invert: true });
new Setting(paneEl).autoWireToggle("watchInternalFileChanges", { invert: true });
});
void addPanel(paneEl, "Compatibility (Remote Database)").then((paneEl) => {
renderObsidianSetting(paneEl, "E2EEAlgorithm", {
new Setting(paneEl).autoWireDropDown("E2EEAlgorithm", {
options: E2EEAlgorithmNames,
});
});
renderObsidianSetting(paneEl, "useDynamicIterationCount", {
new Setting(paneEl).autoWireToggle("useDynamicIterationCount", {
holdValue: true,
onUpdate: visibleOnly(
() =>
@@ -135,15 +134,16 @@ export function panePatches(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElemen
});
void addPanel(paneEl, "Edge case addressing (Database)").then((paneEl) => {
renderObsidianSetting(paneEl, "additionalSuffixOfDatabaseName");
renderObsidianApplyButton(paneEl, "database-suffix");
new Setting(paneEl)
.autoWireText("additionalSuffixOfDatabaseName", { holdValue: true })
.addApplyButton(["additionalSuffixOfDatabaseName"]);
this.addOnSaved("additionalSuffixOfDatabaseName", async (key) => {
Logger("Suffix has been changed. Reopening database...", LOG_LEVEL_NOTICE);
await this.services.databaseEvents.initialiseDatabase();
});
renderObsidianSetting(paneEl, "hashAlg", {
new Setting(paneEl).autoWireDropDown("hashAlg", {
options: {
"": "Old Algorithm",
xxhash32: "xxhash32 (Fast but less collision resistance)",
@@ -157,15 +157,15 @@ export function panePatches(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElemen
});
});
void addPanel(paneEl, "Edge case addressing (Behaviour)").then((paneEl) => {
renderObsidianSetting(paneEl, "doNotSuspendOnFetching");
renderObsidianSetting(paneEl, "doNotDeleteFolder").setClass("wizardHidden");
renderObsidianSetting(paneEl, "processSizeMismatchedFiles");
new Setting(paneEl).autoWireToggle("doNotSuspendOnFetching");
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("doNotDeleteFolder");
new Setting(paneEl).autoWireToggle("processSizeMismatchedFiles");
});
void addPanel(paneEl, "Edge case addressing (Processing)").then((paneEl) => {
renderObsidianSetting(paneEl, "disableWorkerForGeneratingChunks");
new Setting(paneEl).autoWireToggle("disableWorkerForGeneratingChunks");
renderObsidianSetting(paneEl, "processSmallFilesInUIThread", {
new Setting(paneEl).autoWireToggle("processSmallFilesInUIThread", {
onUpdate: visibleOnly(() => this.isConfiguredAs("disableWorkerForGeneratingChunks", false)),
});
});
@@ -173,11 +173,11 @@ export function panePatches(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElemen
// new Setting(paneEl).autoWireToggle("useRequestAPI");
// });
void addPanel(paneEl, "Compatibility (Trouble addressed)").then((paneEl) => {
renderObsidianSetting(paneEl, "disableCheckingConfigMismatch");
new Setting(paneEl).autoWireToggle("disableCheckingConfigMismatch");
});
void addPanel(paneEl, "Remediation").then((paneEl) => {
let dateEl: HTMLSpanElement;
const remediationSetting = new Setting(paneEl)
new Setting(paneEl)
.addText((text) => {
const updateDateText = () => {
if (this.editingSettings.maxMTimeForReflectEvents == 0) {
@@ -212,8 +212,8 @@ export function panePatches(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElemen
updateDateText();
return text;
})
.setAuto("maxMTimeForReflectEvents");
addObsidianApplyButton(remediationSetting, "remediation-reflect-events");
.setAuto("maxMTimeForReflectEvents")
.addApplyButton(["maxMTimeForReflectEvents"]);
this.addOnSaved("maxMTimeForReflectEvents", async (key) => {
const buttons = ["Restart Now", "Later"] as const;
@@ -240,6 +240,6 @@ export function panePatches(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElemen
// .setClass("wizardHidden");
// new Setting(paneEl).autoWireNumeric("maxAgeInEden", { onUpdate: onlyUsingEden }).setClass("wizardHidden");
renderObsidianSetting(paneEl, "enableCompression").setClass("wizardHidden");
new Setting(paneEl).autoWireToggle("enableCompression").setClass("wizardHidden");
});
}

View File

@@ -1,5 +1,5 @@
import { type ConfigPassphraseStore } from "@lib/common/types.ts";
import { renderObsidianApplyButton, renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
import type { PageFunctions } from "./SettingPane.ts";
@@ -21,14 +21,14 @@ export function panePowerUsers(
this.onlyOnCouchDB
).addClass("wizardHidden");
renderObsidianSetting(paneEl, "batch_size", { clampMin: 2, onUpdate: this.onlyOnCouchDB }).setClass(
"wizardHidden"
);
renderObsidianSetting(paneEl, "batches_limit", {
new Setting(paneEl)
.setClass("wizardHidden")
.autoWireNumeric("batch_size", { clampMin: 2, onUpdate: this.onlyOnCouchDB });
new Setting(paneEl).setClass("wizardHidden").autoWireNumeric("batches_limit", {
clampMin: 2,
onUpdate: this.onlyOnCouchDB,
}).setClass("wizardHidden");
renderObsidianSetting(paneEl, "useTimeouts", { onUpdate: this.onlyOnCouchDB }).setClass("wizardHidden");
});
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("useTimeouts", { onUpdate: this.onlyOnCouchDB });
});
void addPanel(paneEl, "Configuration Encryption").then((paneEl) => {
const passphrase_options: Record<ConfigPassphraseStore, string> = {
@@ -37,18 +37,23 @@ export function panePowerUsers(
ASK_AT_LAUNCH: "Ask an passphrase at every launch",
};
renderObsidianSetting(paneEl, "configPassphraseStore", {
options: passphrase_options,
}).setClass("wizardHidden");
new Setting(paneEl)
.setName("Encrypting sensitive configuration items")
.autoWireDropDown("configPassphraseStore", {
options: passphrase_options,
holdValue: true,
})
.setClass("wizardHidden");
renderObsidianSetting(paneEl, "configPassphrase")
new Setting(paneEl)
.autoWireText("configPassphrase", { isPassword: true, holdValue: true })
.setClass("wizardHidden")
.addOnUpdate(() => ({
disabled: !this.isConfiguredAs("configPassphraseStore", "LOCALSTORAGE"),
}));
renderObsidianApplyButton(paneEl, "configuration-encryption").setClass("wizardHidden");
new Setting(paneEl).addApplyButton(["configPassphrase", "configPassphraseStore"]).setClass("wizardHidden");
});
void addPanel(paneEl, "Developer").then((paneEl) => {
renderObsidianSetting(paneEl, "enableDebugTools").setClass("wizardHidden");
new Setting(paneEl).autoWireToggle("enableDebugTools").setClass("wizardHidden");
});
}

View File

@@ -10,7 +10,6 @@ import {
import { Menu, type ButtonComponent } from "@/deps.ts";
import { $msg } from "@lib/common/i18n.ts";
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
import { renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
import type { PageFunctions } from "./SettingPane.ts";
// import { visibleOnly } from "./SettingPane.ts";
@@ -675,7 +674,7 @@ export function paneRemoteConfig(
void addPanel(paneEl, $msg("obsidianLiveSyncSettingTab.titleNotification"), () => {}).then((paneEl) => {
paneEl.addClass("wizardHidden");
renderObsidianSetting(paneEl, "notifyThresholdOfRemoteStorageSize", {}).setClass("wizardHidden");
new Setting(paneEl).autoWireNumeric("notifyThresholdOfRemoteStorageSize", {}).setClass("wizardHidden");
});
// new Setting(paneEl).setClass("wizardOnly").addButton((button) =>

View File

@@ -4,7 +4,6 @@ import MultipleRegExpControl from "./MultipleRegExpControl.svelte";
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
import { mount } from "svelte";
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
import { renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
import type { PageFunctions } from "./SettingPane.ts";
import { visibleOnly } from "./SettingPane.ts";
export function paneSelector(this: ObsidianLiveSyncSettingTab, paneEl: HTMLElement, { addPanel }: PageFunctions): void {
@@ -47,12 +46,12 @@ export function paneSelector(this: ObsidianLiveSyncSettingTab, paneEl: HTMLEleme
},
},
});
renderObsidianSetting(paneEl, "syncMaxSizeInMB", { clampMin: 0 }).setClass("wizardHidden");
new Setting(paneEl).setClass("wizardHidden").autoWireNumeric("syncMaxSizeInMB", { clampMin: 0 });
renderObsidianSetting(paneEl, "useIgnoreFiles").setClass("wizardHidden");
renderObsidianSetting(paneEl, "ignoreFiles", {
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("useIgnoreFiles");
new Setting(paneEl).setClass("wizardHidden").autoWireTextArea("ignoreFiles", {
onUpdate: visibleOnly(() => this.isConfiguredAs("useIgnoreFiles", true)),
}).setClass("wizardHidden");
});
});
void addPanel(paneEl, "Hidden Files", undefined, undefined, LEVEL_ADVANCED).then((paneEl) => {
const targetPatternSetting = new Setting(paneEl)

View File

@@ -15,7 +15,6 @@ import { DEFAULT_SETTINGS } from "@lib/common/types.ts";
import { request } from "@/deps.ts";
import { SetupManager, UserMode } from "@/modules/features/SetupManager.ts";
import { LiveSyncError } from "@lib/common/LSError.ts";
import { renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
export function paneSetup(
this: ObsidianLiveSyncSettingTab,
paneEl: HTMLElement,
@@ -108,9 +107,10 @@ export function paneSetup(
});
void addPanel(paneEl, $msg("obsidianLiveSyncSettingTab.titleExtraFeatures")).then((paneEl) => {
renderObsidianSetting(paneEl, "useAdvancedMode");
renderObsidianSetting(paneEl, "usePowerUserMode");
renderObsidianSetting(paneEl, "useEdgeCaseMode");
new Setting(paneEl).autoWireToggle("useAdvancedMode");
new Setting(paneEl).autoWireToggle("usePowerUserMode");
new Setting(paneEl).autoWireToggle("useEdgeCaseMode");
this.addOnSaved("useAdvancedMode", () => this.display());
this.addOnSaved("usePowerUserMode", () => this.display());

View File

@@ -2,7 +2,6 @@ import { type ObsidianLiveSyncSettings, LOG_LEVEL_NOTICE, REMOTE_COUCHDB, LEVEL_
import { Logger } from "@lib/common/logger.ts";
import { $msg } from "@lib/common/i18n.ts";
import { LiveSyncSetting as Setting } from "./LiveSyncSetting.ts";
import { renderObsidianApplyButton, renderObsidianSetting } from "./ObsidianSettingRenderer.ts";
import { EVENT_REQUEST_COPY_SETUP_URI, eventHub } from "@/common/events.ts";
import type { ObsidianLiveSyncSettingTab } from "./ObsidianLiveSyncSettingTab.ts";
import type { PageFunctions } from "./SettingPane.ts";
@@ -32,16 +31,18 @@ export function paneSyncSettings(
DISABLE: $msg("obsidianLiveSyncSettingTab.optionDisableAllAutomatic"),
};
renderObsidianSetting(paneEl, "preset", {
options: options,
holdValue: true,
}).addButton((button) => {
button.setButtonText($msg("obsidianLiveSyncSettingTab.btnApply"));
button.onClick(async () => {
// await this.saveSettings(["preset"]);
await this.saveAllDirtySettings();
new Setting(paneEl)
.autoWireDropDown("preset", {
options: options,
holdValue: true,
})
.addButton((button) => {
button.setButtonText($msg("obsidianLiveSyncSettingTab.btnApply"));
button.onClick(async () => {
// await this.saveSettings(["preset"]);
await this.saveAllDirtySettings();
});
});
});
this.addOnSaved("preset", async (currentPreset) => {
if (currentPreset == "") {
@@ -135,7 +136,7 @@ export function paneSyncSettings(
const onlyOnNonLiveSync = visibleOnly(() => !this.isConfiguredAs("syncMode", "LIVESYNC"));
const onlyOnPeriodic = visibleOnly(() => this.isConfiguredAs("syncMode", "PERIODIC"));
const optionsSyncMode: Record<string, string> =
const optionsSyncMode =
this.editingSettings.remoteType == REMOTE_COUCHDB
? {
ONEVENTS: $msg("obsidianLiveSyncSettingTab.optionOnEvents"),
@@ -147,9 +148,12 @@ export function paneSyncSettings(
PERIODIC: $msg("obsidianLiveSyncSettingTab.optionPeriodicAndEvents"),
};
renderObsidianSetting(paneEl, "syncMode", {
options: optionsSyncMode,
}).setClass("wizardHidden");
new Setting(paneEl)
.autoWireDropDown("syncMode", {
//@ts-ignore
options: optionsSyncMode,
})
.setClass("wizardHidden");
this.addOnSaved("syncMode", async (value) => {
this.editingSettings.liveSync = false;
this.editingSettings.periodicReplication = false;
@@ -163,28 +167,32 @@ export function paneSyncSettings(
await this.services.control.applySettings();
});
renderObsidianSetting(paneEl, "periodicReplicationInterval", {
clampMax: 5000,
onUpdate: onlyOnPeriodic,
}).setClass("wizardHidden");
new Setting(paneEl)
.autoWireNumeric("periodicReplicationInterval", {
clampMax: 5000,
onUpdate: onlyOnPeriodic,
})
.setClass("wizardHidden");
renderObsidianSetting(paneEl, "syncMinimumInterval", {
new Setting(paneEl).autoWireNumeric("syncMinimumInterval", {
onUpdate: onlyOnNonLiveSync,
});
renderObsidianSetting(paneEl, "syncOnSave", { onUpdate: onlyOnNonLiveSync }).setClass("wizardHidden");
renderObsidianSetting(paneEl, "syncOnEditorSave", { onUpdate: onlyOnNonLiveSync }).setClass("wizardHidden");
renderObsidianSetting(paneEl, "syncOnFileOpen", { onUpdate: onlyOnNonLiveSync }).setClass("wizardHidden");
renderObsidianSetting(paneEl, "syncOnStart", { onUpdate: onlyOnNonLiveSync }).setClass("wizardHidden");
renderObsidianSetting(paneEl, "syncAfterMerge", { onUpdate: onlyOnNonLiveSync }).setClass("wizardHidden");
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("syncOnSave", { onUpdate: onlyOnNonLiveSync });
new Setting(paneEl)
.setClass("wizardHidden")
.autoWireToggle("syncOnEditorSave", { onUpdate: onlyOnNonLiveSync });
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("syncOnFileOpen", { onUpdate: onlyOnNonLiveSync });
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("syncOnStart", { onUpdate: onlyOnNonLiveSync });
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("syncAfterMerge", { onUpdate: onlyOnNonLiveSync });
// Desktop app only, and only for the sync modes that keep a background replication channel
// (LiveSync and Periodic). Ignored on mobile, where suspending preserves battery. The
// visibility predicate mirrors the runtime guard in ModuleObsidianEvents.
if (!this.services.API.isMobile()) {
renderObsidianSetting(paneEl, "keepReplicationActiveInBackground", {
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("keepReplicationActiveInBackground", {
onUpdate: visibleOnly(
() => this.isConfiguredAs("syncMode", "LIVESYNC") || this.isConfiguredAs("syncMode", "PERIODIC")
),
}).setClass("wizardHidden");
});
}
});
@@ -195,15 +203,15 @@ export function paneSyncSettings(
visibleOnly(() => !this.isConfiguredAs("syncMode", "LIVESYNC"))
).then((paneEl) => {
paneEl.addClass("wizardHidden");
renderObsidianSetting(paneEl, "batchSave").setClass("wizardHidden");
renderObsidianSetting(paneEl, "batchSaveMinimumDelay", {
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("batchSave");
new Setting(paneEl).setClass("wizardHidden").autoWireNumeric("batchSaveMinimumDelay", {
acceptZero: true,
onUpdate: visibleOnly(() => this.isConfiguredAs("batchSave", true)),
}).setClass("wizardHidden");
renderObsidianSetting(paneEl, "batchSaveMaximumDelay", {
});
new Setting(paneEl).setClass("wizardHidden").autoWireNumeric("batchSaveMaximumDelay", {
acceptZero: true,
onUpdate: visibleOnly(() => this.isConfiguredAs("batchSave", true)),
}).setClass("wizardHidden");
});
});
void addPanel(
@@ -214,9 +222,9 @@ export function paneSyncSettings(
LEVEL_ADVANCED
).then((paneEl) => {
paneEl.addClass("wizardHidden");
renderObsidianSetting(paneEl, "trashInsteadDelete").setClass("wizardHidden");
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("trashInsteadDelete");
renderObsidianSetting(paneEl, "doNotDeleteFolder").setClass("wizardHidden");
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("doNotDeleteFolder");
});
void addPanel(
paneEl,
@@ -227,11 +235,11 @@ export function paneSyncSettings(
).then((paneEl) => {
paneEl.addClass("wizardHidden");
renderObsidianSetting(paneEl, "resolveConflictsByNewerFile").setClass("wizardHidden");
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("resolveConflictsByNewerFile");
renderObsidianSetting(paneEl, "checkConflictOnlyOnOpen").setClass("wizardHidden");
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("checkConflictOnlyOnOpen");
renderObsidianSetting(paneEl, "showMergeDialogOnlyOnActive").setClass("wizardHidden");
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("showMergeDialogOnlyOnActive");
});
void addPanel(
@@ -242,12 +250,11 @@ export function paneSyncSettings(
LEVEL_ADVANCED
).then((paneEl) => {
paneEl.addClass("wizardHidden");
renderObsidianSetting(paneEl, "settingSyncFile");
renderObsidianApplyButton(paneEl, "setting-sync-file");
new Setting(paneEl).autoWireText("settingSyncFile", { holdValue: true }).addApplyButton(["settingSyncFile"]);
renderObsidianSetting(paneEl, "writeCredentialsForSettingSync");
new Setting(paneEl).autoWireToggle("writeCredentialsForSettingSync");
renderObsidianSetting(paneEl, "notifyAllSettingSyncFile");
new Setting(paneEl).autoWireToggle("notifyAllSettingSyncFile");
});
void addPanel(
@@ -306,14 +313,14 @@ export function paneSyncSettings(
});
}
renderObsidianSetting(paneEl, "suppressNotifyHiddenFilesChange", {}).setClass("wizardHidden");
renderObsidianSetting(paneEl, "syncInternalFilesBeforeReplication", {
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("suppressNotifyHiddenFilesChange", {});
new Setting(paneEl).setClass("wizardHidden").autoWireToggle("syncInternalFilesBeforeReplication", {
onUpdate: visibleOnly(() => this.isConfiguredAs("watchInternalFileChanges", true)),
}).setClass("wizardHidden");
});
renderObsidianSetting(paneEl, "syncInternalFilesInterval", {
new Setting(paneEl).setClass("wizardHidden").autoWireNumeric("syncInternalFilesInterval", {
clampMin: 10,
acceptZero: true,
}).setClass("wizardHidden");
});
});
}

View File

@@ -1,11 +1,5 @@
import { $msg } from "@lib/common/i18n";
import {
LEVEL_ADVANCED,
LEVEL_EDGE_CASE,
LEVEL_POWER_USER,
type ConfigLevel,
type SettingDefinition,
} from "@lib/common/types";
import { LEVEL_ADVANCED, LEVEL_EDGE_CASE, LEVEL_POWER_USER, type ConfigLevel } from "@lib/common/types";
import type { AllSettingItemKey, AllSettings } from "./settingConstants";
export const combineOnUpdate = (func1: OnUpdateFunc, func2: OnUpdateFunc): OnUpdateFunc => {
@@ -83,7 +77,6 @@ export type AutoWireOption = {
invert?: boolean;
onUpdate?: OnUpdateFunc;
obsolete?: boolean;
settingDefinition?: SettingDefinition<AllSettingItemKey>;
};
export function findAttrFromParent(el: HTMLElement, attr: string): string {

1
test/bench-network/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
bench-results/

View File

@@ -0,0 +1,8 @@
FROM alpine:3.22
RUN apk add --no-cache iproute2
COPY test/bench-network/netem-smoke.sh /usr/local/bin/livesync-netem-smoke
RUN chmod +x /usr/local/bin/livesync-netem-smoke
CMD ["livesync-netem-smoke"]

View File

@@ -0,0 +1,35 @@
# syntax=docker/dockerfile:1
FROM node:24-slim
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl unzip python3 make g++ iproute2 \
&& rm -rf /var/lib/apt/lists/*
ENV DENO_INSTALL=/usr/local
RUN curl -fsSL https://deno.land/install.sh | sh
WORKDIR /workspace
COPY package.json package-lock.json ./
COPY src/apps/cli/package.json ./src/apps/cli/package.json
COPY src/apps/webapp/package.json ./src/apps/webapp/package.json
COPY src/apps/webpeer/package.json ./src/apps/webpeer/package.json
RUN npm ci
COPY . .
RUN npm run build -w self-hosted-livesync-cli
WORKDIR /workspace/src/apps/cli/testdeno
RUN deno cache --lock=deno.lock \
bench-network-cases.ts \
bench-latency-sweep.ts \
bench-p2p-split-node.ts \
bench-p2p.ts \
bench-couchdb.ts
COPY test/bench-network/run-bench.sh /usr/local/bin/run-livesync-bench
RUN chmod +x /usr/local/bin/run-livesync-bench
CMD ["run-livesync-bench"]

View File

@@ -0,0 +1,10 @@
# syntax=docker/dockerfile:1
FROM alpine:3.22
RUN apk add --no-cache iproute2 socat
COPY test/bench-network/netem-tcp-shim.sh /usr/local/bin/livesync-netem-tcp-shim
RUN chmod +x /usr/local/bin/livesync-netem-tcp-shim
CMD ["livesync-netem-tcp-shim"]

View File

@@ -0,0 +1,263 @@
# Network benchmark package
This directory packages the CLI benchmark cases with Docker Compose. It is
intended for reproducible local benchmark runs where CouchDB, the Nostr
signalling relay, optional TURN, and the benchmark runner are fixed by the
Compose file.
## Quick smoke run
From the repository root:
```bash
docker compose -f test/bench-network/compose.yml run --rm bench-runner
```
By default this runs:
- `couchdb-baseline`
- `p2p-direct-local`
The dataset is intentionally small by default. Results are written to
`test/bench-network/bench-results/`.
## GitHub Actions smoke run
`.github/workflows/cli-p2p-compose-smoke.yml` provides a manual
`workflow_dispatch` smoke run for the same Compose package. It is intentionally
not a required check yet, because WebRTC peer discovery can still be slow or
environment-sensitive on GitHub-hosted runners. Keep the dataset small and use
the uploaded JSON artefact to inspect whether failures are caused by peer
discovery, synchronisation, CouchDB startup, or Docker networking.
## Select cases
```bash
BENCH_CASES=couchdb-baseline,p2p-direct-local,p2p-user-turn \
docker compose -f test/bench-network/compose.yml --profile turn run --rm bench-runner
```
Available local cases:
- `couchdb-baseline`
- `p2p-direct-local`
- `couchdb-tethering-vpn-proxy`
- `couchdb-netem-home-wifi`
- `couchdb-netem-tethering-vpn`
- `p2p-smartphone-vpn-direct`
- `p2p-user-turn`
Set `BENCH_REPEAT_COUNT` to run each selected case more than once. Repeated
results are written with suffixes such as `-r01`, `-r02`, and `-r03`, and the
summary records the repeat index for each run.
`p2p-smartphone-vpn-direct` is a structural case name. When it is run inside
this Compose package it is not a real smartphone tethering/VPN measurement; it
uses the local Compose network. Use it only for wiring checks unless the runner
is executed in an actual tethered/VPN environment.
## Comparison model
The primary local comparison is between a remote-database path and a direct P2P
path:
| Case | Data path | What is measured | What is not measured |
| ------------------ | ------------------------------------------- | ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------- |
| `couchdb-baseline` | Device A -> CouchDB -> Device B | Two one-shot CLI synchronisation commands through a local HTTP latency proxy | Real WAN jitter, packet loss, bandwidth limits, VPN encapsulation, and server contention |
| `p2p-direct-local` | Device A -> Device B after Nostr signalling | One CLI P2P synchronisation command over WebRTC DataChannel with TURN disabled | Public relay operation, mobile carrier behaviour, TURN relay throughput, and first-peer discovery latency |
Use the CouchDB result as the remote-store baseline and the P2P result as the
direct-transfer comparison. The Nostr relay is used for signalling in the P2P
case, but synchronised note content is transferred over the WebRTC DataChannel.
The P2P result JSON records the selected WebRTC ICE candidate pair when the CLI
can collect it from `RTCPeerConnection.getStats()`. Interpret P2P paths from
the recorded candidate types rather than from TURN configuration alone. Do not
report P2P runs as Tier 2 constrained-network measurements until host and
client are captured under an equivalent shaped topology.
## Dataset and latency controls
```bash
BENCH_MD_FILE_COUNT=100 \
BENCH_MD_MIN_SIZE_BYTES=512 \
BENCH_MD_MAX_SIZE_BYTES=2048 \
BENCH_BIN_FILE_COUNT=25 \
BENCH_BIN_SIZE_BYTES=8192 \
BENCH_COUCHDB_RTT_MS=20 \
BENCH_PEERS_TIMEOUT=60 \
docker compose -f test/bench-network/compose.yml run --rm bench-runner
```
The current CouchDB latency model is the existing HTTP proxy inside
`bench-couchdb.ts`. It models a remote database path with additional request
latency, but it does not model packet loss, jitter, MTU, bandwidth limits,
bufferbloat, or VPN encapsulation.
For P2P runs, `BENCH_PEERS_TIMEOUT` is passed to `p2p-peers`. That command waits
for the requested observation window before printing discovered peers, so the
reported peer discovery command time should not be read as first-peer latency.
## Latency sweep
To run P2P once and CouchDB at several requested RTT values:
```bash
BENCH_COMMAND=latency-sweep \
BENCH_SWEEP_RTT_MS=20,50,100,150,300 \
BENCH_MD_FILE_COUNT=100 \
BENCH_MD_MIN_SIZE_BYTES=512 \
BENCH_MD_MAX_SIZE_BYTES=2048 \
BENCH_BIN_FILE_COUNT=25 \
BENCH_BIN_SIZE_BYTES=8192 \
BENCH_SYNC_TIMEOUT=300 \
BENCH_PEERS_TIMEOUT=60 \
docker compose -f test/bench-network/compose.yml run --rm bench-runner
```
This sweep is useful for finding where the remote CouchDB path falls behind the
local direct P2P path in the current HTTP-proxy latency model. It should not be
presented as a full smartphone/VPN model.
## Network emulation smoke
The optional `netem` profile checks whether a Linux runner can apply traffic
shaping inside a Compose-managed container. This is a fixture smoke test for a
second-tier simulation design; it does not produce synchronisation performance
results by itself.
```bash
docker compose -f test/bench-network/compose.yml --profile netem run --rm netem-smoke
```
The smoke writes `tc qdisc`, route, and interface details under
`test/bench-network/bench-results/`. Profile parameters can be overridden:
```bash
NETEM_PROFILE=tethering-vpn \
NETEM_DELAY_MS=140 \
NETEM_JITTER_MS=50 \
NETEM_LOSS_PERCENT=1.0 \
NETEM_BANDWIDTH_MBIT=10 \
NETEM_MTU=1380 \
docker compose -f test/bench-network/compose.yml --profile netem run --rm netem-smoke
```
## Split-container P2P emulation
The optional `p2p-split` profile runs the P2P host and client in separate
Compose services. Each service can apply `tc netem` to its own egress interface
and the client result records the selected WebRTC ICE candidate pair.
```bash
BENCH_MD_FILE_COUNT=2 \
BENCH_BIN_FILE_COUNT=1 \
BENCH_PEERS_TIMEOUT=10 \
BENCH_SPLIT_RUN_ID="$(date -u +%Y%m%d%H%M%S)" \
docker compose -f test/bench-network/compose.yml --profile p2p-split up \
--abort-on-container-exit --exit-code-from p2p-split-client \
p2p-split-host p2p-split-client
```
By default this uses the `home-wifi` profile (`20 ms` delay, `5 ms` jitter,
`0.1%` loss, `100 Mbit`, and `1500` MTU) on both P2P containers. Override the
same `NETEM_*` variables used by the TCP shim to model a stricter profile.
```bash
BENCH_MD_FILE_COUNT=100 \
BENCH_MD_MIN_SIZE_BYTES=512 \
BENCH_MD_MAX_SIZE_BYTES=2048 \
BENCH_BIN_FILE_COUNT=25 \
BENCH_BIN_SIZE_BYTES=8192 \
BENCH_PEERS_TIMEOUT=60 \
BENCH_SYNC_TIMEOUT=420 \
BENCH_SPLIT_RUN_ID="$(date -u +%Y%m%d%H%M%S)" \
BENCH_NETWORK_PROFILE=tethering-vpn \
NETEM_PROFILE=tethering-vpn \
NETEM_DELAY_MS=140 \
NETEM_JITTER_MS=50 \
NETEM_LOSS_PERCENT=1.0 \
NETEM_BANDWIDTH_MBIT=10 \
NETEM_MTU=1380 \
docker compose -f test/bench-network/compose.yml --profile p2p-split up \
--abort-on-container-exit --exit-code-from p2p-split-client \
p2p-split-host p2p-split-client
```
This is a Linux-only manual benchmark fixture, not a required pull-request CI
job. It shapes each P2P container's egress path, including signalling traffic,
and should be reported separately from the CouchDB TCP-shim measurements. The
result JSON includes `ok: true` for completed runs; failed runs still write a
summary with `ok: false` and a `failure` object before returning a non-zero
exit code.
Remove the shared work volume between repeated manual runs when you do not use
a unique `BENCH_SPLIT_RUN_ID`:
```bash
docker compose -f test/bench-network/compose.yml --profile p2p-split down --volumes
```
## P2P Signalling-Only Emulation
The optional `signalling-shim` profile shapes only the Nostr signalling relay
path. The P2P host and client run in the benchmark runner as usual, and the
configured relay URL points at a TCP netem shim in front of `nostr-relay`.
This is the preferred fixture when evaluating the hypothesis that P2P avoids a
constrained remote database data path while still depending on a signalling
server for rendezvous.
```bash
BENCH_CASES=p2p-signalling-netem-home-wifi \
docker compose -f test/bench-network/compose.yml --profile signalling-shim run --rm \
bench-runner-signalling-shim
```
For a stricter signalling path:
```bash
NETEM_PROFILE=tethering-vpn \
NETEM_DELAY_MS=140 \
NETEM_JITTER_MS=50 \
NETEM_LOSS_PERCENT=1.0 \
NETEM_BANDWIDTH_MBIT=10 \
NETEM_MTU=1380 \
BENCH_CASES=p2p-signalling-netem-tethering-vpn \
docker compose -f test/bench-network/compose.yml --profile signalling-shim run --rm \
bench-runner-signalling-shim
```
Use this separately from `p2p-split`. The `p2p-split` profile shapes each peer's
egress path, so it constrains both signalling and the selected WebRTC data
path. The `signalling-shim` profile constrains only relay access, which keeps
it focused on peer-to-signalling-server reachability rather than peer-to-peer
note-data transfer.
## Shimmed CouchDB benchmark
The optional `shim` profile runs a CouchDB benchmark through a TCP forwarding
container that applies `tc netem`. This is a manual Tier 2 synchronisation
measurement path; it is intentionally separate from required pull-request CI.
```bash
docker compose -f test/bench-network/compose.yml --profile shim run --rm bench-runner-shim
```
The default profile is `home-wifi`. A smartphone/VPN-like profile can be
requested by overriding both the shim parameters and the benchmark case:
```bash
NETEM_PROFILE=tethering-vpn \
NETEM_DELAY_MS=140 \
NETEM_JITTER_MS=50 \
NETEM_LOSS_PERCENT=1.0 \
NETEM_BANDWIDTH_MBIT=10 \
NETEM_MTU=1380 \
BENCH_CASES=couchdb-netem-tethering-vpn \
docker compose -f test/bench-network/compose.yml --profile shim run --rm bench-runner-shim
```
The benchmark result records `simulationTier`, `networkProfile`, and
`networkModel`. The shim also writes its applied `tc qdisc`, route, and
interface state under `test/bench-network/bench-results/`.
This shim currently measures the CouchDB path only. It does not shape or verify
the WebRTC P2P data path.

View File

@@ -0,0 +1,329 @@
services:
couchdb:
image: couchdb:3.5.0
environment:
COUCHDB_USER: ${BENCH_COUCHDB_USER:-admin}
COUCHDB_PASSWORD: ${BENCH_COUCHDB_PASSWORD:-testpassword}
COUCHDB_SINGLE_NODE: "true"
healthcheck:
test:
[
"CMD-SHELL",
"curl -fsS -u ${BENCH_COUCHDB_USER:-admin}:${BENCH_COUCHDB_PASSWORD:-testpassword} http://127.0.0.1:5984/_up >/dev/null",
]
interval: 2s
timeout: 5s
retries: 30
nostr-relay:
image: ghcr.io/hoytech/strfry:latest
entrypoint: sh
command:
- -lc
- |
cat > /tmp/strfry.conf <<'EOF'
db = "./strfry-db/"
relay {
bind = "0.0.0.0"
port = 7777
nofiles = 65536
info {
name = "livesync bench relay"
description = "local relay for livesync compose benchmarks"
}
maxWebsocketPayloadSize = 131072
autoPingSeconds = 55
writePolicy {
plugin = ""
}
}
EOF
exec /app/strfry --config /tmp/strfry.conf relay
tmpfs:
- /app/strfry-db:rw,size=256m
healthcheck:
test: ["CMD-SHELL", "nc -z 127.0.0.1 7777"]
interval: 2s
timeout: 5s
retries: 30
coturn:
image: coturn/coturn:latest
command:
- --log-file=stdout
- --listening-port=3478
- --user=${BENCH_TURN_USERNAME:-testuser}:${BENCH_TURN_CREDENTIAL:-testpass}
- --realm=${BENCH_TURN_REALM:-livesync.test}
profiles:
- turn
bench-runner:
build:
context: ../..
dockerfile: test/bench-network/Dockerfile.runner
depends_on:
couchdb:
condition: service_healthy
nostr-relay:
condition: service_healthy
environment:
BENCH_COMMAND: ${BENCH_COMMAND:-cases}
BENCH_CASES: ${BENCH_CASES:-couchdb-baseline,p2p-direct-local}
BENCH_REPEAT_COUNT: ${BENCH_REPEAT_COUNT:-1}
BENCH_CASES_ROOT: /workspace/src/apps/cli/testdeno/bench-results
BENCH_SWEEP_ROOT: /workspace/src/apps/cli/testdeno/bench-results
BENCH_SWEEP_RTT_MS: ${BENCH_SWEEP_RTT_MS:-20,50,100,150,300}
BENCH_SWEEP_INCLUDE_P2P: ${BENCH_SWEEP_INCLUDE_P2P:-true}
BENCH_COUCHDB_MANAGED: "false"
BENCH_COUCHDB_BACKEND_URI: http://couchdb:5984
BENCH_COUCHDB_URI: http://127.0.0.1:15989
BENCH_COUCHDB_USER: ${BENCH_COUCHDB_USER:-admin}
BENCH_COUCHDB_PASSWORD: ${BENCH_COUCHDB_PASSWORD:-testpassword}
BENCH_RELAY: ws://nostr-relay:7777/
BENCH_LOCAL_TURN_SERVERS: turn:coturn:3478
BENCH_MD_FILE_COUNT: ${BENCH_MD_FILE_COUNT:-20}
BENCH_MD_MIN_SIZE_BYTES: ${BENCH_MD_MIN_SIZE_BYTES:-512}
BENCH_MD_MAX_SIZE_BYTES: ${BENCH_MD_MAX_SIZE_BYTES:-2048}
BENCH_BIN_FILE_COUNT: ${BENCH_BIN_FILE_COUNT:-5}
BENCH_BIN_SIZE_BYTES: ${BENCH_BIN_SIZE_BYTES:-8192}
BENCH_COUCHDB_RTT_MS: ${BENCH_COUCHDB_RTT_MS:-20}
BENCH_TETHERING_VPN_RTT_MS: ${BENCH_TETHERING_VPN_RTT_MS:-120}
BENCH_SYNC_TIMEOUT: ${BENCH_SYNC_TIMEOUT:-300}
BENCH_PEERS_TIMEOUT: ${BENCH_PEERS_TIMEOUT:-60}
LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS: ${LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS:-60000}
BENCH_LIVESYNC_TEST_TEE: ${BENCH_LIVESYNC_TEST_TEE:-0}
volumes:
- ./bench-results:/workspace/src/apps/cli/testdeno/bench-results
couchdb-shim:
build:
context: ../..
dockerfile: test/bench-network/Dockerfile.shim
profiles:
- shim
depends_on:
couchdb:
condition: service_healthy
cap_add:
- NET_ADMIN
environment:
NETEM_PROFILE: ${NETEM_PROFILE:-home-wifi}
NETEM_INTERFACE: ${NETEM_INTERFACE:-eth0}
NETEM_DELAY_MS: ${NETEM_DELAY_MS:-20}
NETEM_JITTER_MS: ${NETEM_JITTER_MS:-5}
NETEM_LOSS_PERCENT: ${NETEM_LOSS_PERCENT:-0.1}
NETEM_BANDWIDTH_MBIT: ${NETEM_BANDWIDTH_MBIT:-100}
NETEM_MTU: ${NETEM_MTU:-1500}
NETEM_RESULT_ROOT: /bench-results
SHIM_LISTEN_PORT: 5984
SHIM_TARGET_HOST: couchdb
SHIM_TARGET_PORT: 5984
volumes:
- ./bench-results:/bench-results
healthcheck:
test: ["CMD-SHELL", "nc -z 127.0.0.1 5984"]
interval: 2s
timeout: 5s
retries: 30
bench-runner-shim:
build:
context: ../..
dockerfile: test/bench-network/Dockerfile.runner
profiles:
- shim
depends_on:
couchdb-shim:
condition: service_healthy
environment:
BENCH_COMMAND: ${BENCH_COMMAND:-cases}
BENCH_CASES: ${BENCH_CASES:-couchdb-netem-home-wifi}
BENCH_REPEAT_COUNT: ${BENCH_REPEAT_COUNT:-1}
BENCH_CASES_ROOT: /workspace/src/apps/cli/testdeno/bench-results
BENCH_SWEEP_ROOT: /workspace/src/apps/cli/testdeno/bench-results
BENCH_COUCHDB_MANAGED: "false"
BENCH_COUCHDB_BACKEND_URI: http://couchdb-shim:5984
BENCH_SHIM_COUCHDB_URI: http://couchdb-shim:5984
BENCH_COUCHDB_URI: http://127.0.0.1:15989
BENCH_COUCHDB_USER: ${BENCH_COUCHDB_USER:-admin}
BENCH_COUCHDB_PASSWORD: ${BENCH_COUCHDB_PASSWORD:-testpassword}
BENCH_MD_FILE_COUNT: ${BENCH_MD_FILE_COUNT:-20}
BENCH_MD_MIN_SIZE_BYTES: ${BENCH_MD_MIN_SIZE_BYTES:-512}
BENCH_MD_MAX_SIZE_BYTES: ${BENCH_MD_MAX_SIZE_BYTES:-2048}
BENCH_BIN_FILE_COUNT: ${BENCH_BIN_FILE_COUNT:-5}
BENCH_BIN_SIZE_BYTES: ${BENCH_BIN_SIZE_BYTES:-8192}
BENCH_COUCHDB_RTT_MS: ${BENCH_COUCHDB_RTT_MS:-1}
BENCH_SYNC_TIMEOUT: ${BENCH_SYNC_TIMEOUT:-300}
BENCH_LIVESYNC_TEST_TEE: ${BENCH_LIVESYNC_TEST_TEE:-0}
volumes:
- ./bench-results:/workspace/src/apps/cli/testdeno/bench-results
p2p-signalling-shim:
build:
context: ../..
dockerfile: test/bench-network/Dockerfile.shim
profiles:
- signalling-shim
depends_on:
nostr-relay:
condition: service_healthy
cap_add:
- NET_ADMIN
environment:
NETEM_PROFILE: ${NETEM_PROFILE:-home-wifi}
NETEM_INTERFACE: ${NETEM_INTERFACE:-eth0}
NETEM_DELAY_MS: ${NETEM_DELAY_MS:-20}
NETEM_JITTER_MS: ${NETEM_JITTER_MS:-5}
NETEM_LOSS_PERCENT: ${NETEM_LOSS_PERCENT:-0.1}
NETEM_BANDWIDTH_MBIT: ${NETEM_BANDWIDTH_MBIT:-100}
NETEM_MTU: ${NETEM_MTU:-1500}
NETEM_RESULT_ROOT: /bench-results
SHIM_LISTEN_PORT: 7777
SHIM_TARGET_HOST: nostr-relay
SHIM_TARGET_PORT: 7777
volumes:
- ./bench-results:/bench-results
healthcheck:
test: ["CMD-SHELL", "nc -z 127.0.0.1 7777"]
interval: 2s
timeout: 5s
retries: 30
bench-runner-signalling-shim:
build:
context: ../..
dockerfile: test/bench-network/Dockerfile.runner
profiles:
- signalling-shim
depends_on:
p2p-signalling-shim:
condition: service_healthy
environment:
BENCH_COMMAND: ${BENCH_COMMAND:-cases}
BENCH_CASES: ${BENCH_CASES:-p2p-signalling-netem-home-wifi}
BENCH_REPEAT_COUNT: ${BENCH_REPEAT_COUNT:-1}
BENCH_CASES_ROOT: /workspace/src/apps/cli/testdeno/bench-results
BENCH_SIGNAL_SHIM_RELAY: ws://p2p-signalling-shim:7777/
BENCH_MD_FILE_COUNT: ${BENCH_MD_FILE_COUNT:-20}
BENCH_MD_MIN_SIZE_BYTES: ${BENCH_MD_MIN_SIZE_BYTES:-512}
BENCH_MD_MAX_SIZE_BYTES: ${BENCH_MD_MAX_SIZE_BYTES:-2048}
BENCH_BIN_FILE_COUNT: ${BENCH_BIN_FILE_COUNT:-5}
BENCH_BIN_SIZE_BYTES: ${BENCH_BIN_SIZE_BYTES:-8192}
BENCH_SYNC_TIMEOUT: ${BENCH_SYNC_TIMEOUT:-300}
BENCH_PEERS_TIMEOUT: ${BENCH_PEERS_TIMEOUT:-60}
LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS: ${LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS:-60000}
BENCH_LIVESYNC_TEST_TEE: ${BENCH_LIVESYNC_TEST_TEE:-0}
volumes:
- ./bench-results:/workspace/src/apps/cli/testdeno/bench-results
p2p-split-host:
build:
context: ../..
dockerfile: test/bench-network/Dockerfile.runner
profiles:
- p2p-split
depends_on:
nostr-relay:
condition: service_healthy
cap_add:
- NET_ADMIN
environment:
BENCH_COMMAND: p2p-split-node
BENCH_P2P_SPLIT_ROLE: host
BENCH_SPLIT_RUN_ID: ${BENCH_SPLIT_RUN_ID:-bench-split-run}
BENCH_SPLIT_WORK_ROOT: /p2p-work
BENCH_SPLIT_RESULT_ROOT: /workspace/src/apps/cli/testdeno/bench-results
BENCH_RELAY: ws://nostr-relay:7777/
BENCH_APP_ID: ${BENCH_APP_ID:-self-hosted-livesync-cli-benchmark}
BENCH_ROOM_ID: ${BENCH_ROOM_ID:-bench-split-room}
BENCH_PASSPHRASE: ${BENCH_PASSPHRASE:-bench-split-passphrase}
BENCH_TURN_SERVERS: ${BENCH_TURN_SERVERS:-}
BENCH_MD_FILE_COUNT: ${BENCH_MD_FILE_COUNT:-20}
BENCH_MD_MIN_SIZE_BYTES: ${BENCH_MD_MIN_SIZE_BYTES:-512}
BENCH_MD_MAX_SIZE_BYTES: ${BENCH_MD_MAX_SIZE_BYTES:-2048}
BENCH_BIN_FILE_COUNT: ${BENCH_BIN_FILE_COUNT:-5}
BENCH_BIN_SIZE_BYTES: ${BENCH_BIN_SIZE_BYTES:-8192}
BENCH_SYNC_TIMEOUT: ${BENCH_SYNC_TIMEOUT:-300}
BENCH_PEERS_TIMEOUT: ${BENCH_PEERS_TIMEOUT:-60}
BENCH_NETEM_ENABLED: ${BENCH_NETEM_ENABLED:-1}
BENCH_NETWORK_PROFILE: ${BENCH_NETWORK_PROFILE:-home-wifi}
NETEM_PROFILE: ${NETEM_PROFILE:-home-wifi}
NETEM_INTERFACE: ${NETEM_INTERFACE:-eth0}
NETEM_DELAY_MS: ${NETEM_DELAY_MS:-20}
NETEM_JITTER_MS: ${NETEM_JITTER_MS:-5}
NETEM_LOSS_PERCENT: ${NETEM_LOSS_PERCENT:-0.1}
NETEM_BANDWIDTH_MBIT: ${NETEM_BANDWIDTH_MBIT:-100}
NETEM_MTU: ${NETEM_MTU:-1500}
LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS: ${LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS:-60000}
LIVESYNC_TEST_TEE: ${BENCH_LIVESYNC_TEST_TEE:-0}
volumes:
- p2p-split-work:/p2p-work
- ./bench-results:/workspace/src/apps/cli/testdeno/bench-results
p2p-split-client:
build:
context: ../..
dockerfile: test/bench-network/Dockerfile.runner
profiles:
- p2p-split
depends_on:
nostr-relay:
condition: service_healthy
p2p-split-host:
condition: service_started
cap_add:
- NET_ADMIN
environment:
BENCH_COMMAND: p2p-split-node
BENCH_P2P_SPLIT_ROLE: client
BENCH_SPLIT_RUN_ID: ${BENCH_SPLIT_RUN_ID:-bench-split-run}
BENCH_SPLIT_WORK_ROOT: /p2p-work
BENCH_SPLIT_RESULT_ROOT: /workspace/src/apps/cli/testdeno/bench-results
BENCH_RELAY: ws://nostr-relay:7777/
BENCH_APP_ID: ${BENCH_APP_ID:-self-hosted-livesync-cli-benchmark}
BENCH_ROOM_ID: ${BENCH_ROOM_ID:-bench-split-room}
BENCH_PASSPHRASE: ${BENCH_PASSPHRASE:-bench-split-passphrase}
BENCH_TURN_SERVERS: ${BENCH_TURN_SERVERS:-}
BENCH_SYNC_TIMEOUT: ${BENCH_SYNC_TIMEOUT:-300}
BENCH_PEERS_TIMEOUT: ${BENCH_PEERS_TIMEOUT:-60}
BENCH_NETEM_ENABLED: ${BENCH_NETEM_ENABLED:-1}
BENCH_NETWORK_PROFILE: ${BENCH_NETWORK_PROFILE:-home-wifi}
NETEM_PROFILE: ${NETEM_PROFILE:-home-wifi}
NETEM_INTERFACE: ${NETEM_INTERFACE:-eth0}
NETEM_DELAY_MS: ${NETEM_DELAY_MS:-20}
NETEM_JITTER_MS: ${NETEM_JITTER_MS:-5}
NETEM_LOSS_PERCENT: ${NETEM_LOSS_PERCENT:-0.1}
NETEM_BANDWIDTH_MBIT: ${NETEM_BANDWIDTH_MBIT:-100}
NETEM_MTU: ${NETEM_MTU:-1500}
LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS: ${LIVESYNC_P2P_RELAY_READY_TIMEOUT_MS:-60000}
LIVESYNC_TEST_TEE: ${BENCH_LIVESYNC_TEST_TEE:-0}
volumes:
- p2p-split-work:/p2p-work
- ./bench-results:/workspace/src/apps/cli/testdeno/bench-results
netem-smoke:
build:
context: ../..
dockerfile: test/bench-network/Dockerfile.netem
profiles:
- netem
cap_add:
- NET_ADMIN
environment:
NETEM_PROFILE: ${NETEM_PROFILE:-home-wifi}
NETEM_INTERFACE: ${NETEM_INTERFACE:-eth0}
NETEM_DELAY_MS: ${NETEM_DELAY_MS:-20}
NETEM_JITTER_MS: ${NETEM_JITTER_MS:-5}
NETEM_LOSS_PERCENT: ${NETEM_LOSS_PERCENT:-0.1}
NETEM_BANDWIDTH_MBIT: ${NETEM_BANDWIDTH_MBIT:-100}
NETEM_MTU: ${NETEM_MTU:-1500}
NETEM_RESULT_ROOT: /bench-results
volumes:
- ./bench-results:/bench-results
volumes:
p2p-split-work:

View File

@@ -0,0 +1,71 @@
#!/usr/bin/env sh
set -eu
profile="${NETEM_PROFILE:-home-wifi}"
iface="${NETEM_INTERFACE:-eth0}"
delay_ms="${NETEM_DELAY_MS:-20}"
jitter_ms="${NETEM_JITTER_MS:-5}"
loss_percent="${NETEM_LOSS_PERCENT:-0.1}"
bandwidth_mbit="${NETEM_BANDWIDTH_MBIT:-100}"
mtu="${NETEM_MTU:-1500}"
out_root="${NETEM_RESULT_ROOT:-/bench-results}"
timestamp="$(date -u +%Y%m%d-%H%M%S)"
out_dir="${out_root}/netem-smoke-${timestamp}"
out_file="${out_dir}/summary.json"
mkdir -p "$out_dir"
if ! ip link show "$iface" >/dev/null 2>&1; then
echo "Network interface '$iface' was not found" >&2
ip addr >&2
exit 2
fi
ip link set dev "$iface" mtu "$mtu"
tc qdisc del dev "$iface" root >/dev/null 2>&1 || true
tc qdisc add dev "$iface" root netem \
delay "${delay_ms}ms" "${jitter_ms}ms" \
loss "${loss_percent}%" \
rate "${bandwidth_mbit}mbit"
json_lines() {
awk '
{
gsub(/\\/, "\\\\");
gsub(/"/, "\\\"");
printf "%s \"%s\"", (NR == 1 ? "" : ",\n"), $0;
}
'
}
ip_addr="$(ip addr show "$iface" | json_lines)"
ip_route="$(ip route | json_lines)"
tc_qdisc="$(tc qdisc show dev "$iface" | json_lines)"
cat > "$out_file" <<EOF
{
"simulationTier": 2,
"mode": "netem-smoke",
"profile": "$profile",
"interface": "$iface",
"netem": {
"delayMs": $delay_ms,
"jitterMs": $jitter_ms,
"lossPercent": $loss_percent,
"bandwidthMbit": $bandwidth_mbit,
"mtu": $mtu
},
"ipAddr": [
$ip_addr
],
"ipRoute": [
$ip_route
],
"tcQdisc": [
$tc_qdisc
]
}
EOF
cat "$out_file"
echo "[netem-smoke] result file: $out_file"

View File

@@ -0,0 +1,80 @@
#!/usr/bin/env sh
set -eu
profile="${NETEM_PROFILE:-home-wifi}"
iface="${NETEM_INTERFACE:-eth0}"
delay_ms="${NETEM_DELAY_MS:-20}"
jitter_ms="${NETEM_JITTER_MS:-5}"
loss_percent="${NETEM_LOSS_PERCENT:-0.1}"
bandwidth_mbit="${NETEM_BANDWIDTH_MBIT:-100}"
mtu="${NETEM_MTU:-1500}"
listen_port="${SHIM_LISTEN_PORT:-5984}"
target_host="${SHIM_TARGET_HOST:-couchdb}"
target_port="${SHIM_TARGET_PORT:-5984}"
out_root="${NETEM_RESULT_ROOT:-/bench-results}"
timestamp="$(date -u +%Y%m%d-%H%M%S)"
out_dir="${out_root}/netem-shim-${profile}-${timestamp}"
out_file="${out_dir}/summary.json"
json_lines() {
awk '
{
gsub(/\\/, "\\\\");
gsub(/"/, "\\\"");
printf "%s \"%s\"", (NR == 1 ? "" : ",\n"), $0;
}
'
}
mkdir -p "$out_dir"
if ! ip link show "$iface" >/dev/null 2>&1; then
echo "Network interface '$iface' was not found" >&2
ip addr >&2
exit 2
fi
ip link set dev "$iface" mtu "$mtu"
tc qdisc del dev "$iface" root >/dev/null 2>&1 || true
tc qdisc add dev "$iface" root netem \
delay "${delay_ms}ms" "${jitter_ms}ms" \
loss "${loss_percent}%" \
rate "${bandwidth_mbit}mbit"
ip_addr="$(ip addr show "$iface" | json_lines)"
ip_route="$(ip route | json_lines)"
tc_qdisc="$(tc qdisc show dev "$iface" | json_lines)"
cat > "$out_file" <<EOF
{
"simulationTier": 2,
"mode": "netem-tcp-shim",
"profile": "$profile",
"interface": "$iface",
"listenPort": $listen_port,
"targetHost": "$target_host",
"targetPort": $target_port,
"netem": {
"delayMs": $delay_ms,
"jitterMs": $jitter_ms,
"lossPercent": $loss_percent,
"bandwidthMbit": $bandwidth_mbit,
"mtu": $mtu
},
"ipAddr": [
$ip_addr
],
"ipRoute": [
$ip_route
],
"tcQdisc": [
$tc_qdisc
]
}
EOF
cat "$out_file"
echo "[netem-shim] forwarding 0.0.0.0:${listen_port} to ${target_host}:${target_port}"
echo "[netem-shim] result file: $out_file"
exec socat "TCP-LISTEN:${listen_port},fork,reuseaddr" "TCP:${target_host}:${target_port}"

View File

@@ -0,0 +1,19 @@
#!/usr/bin/env sh
set -eu
case "${BENCH_COMMAND:-cases}" in
cases)
exec deno task bench:cases
;;
latency-sweep)
exec deno task bench:latency-sweep
;;
p2p-split-node)
exec deno task bench:p2p-split-node
;;
*)
echo "Unknown BENCH_COMMAND: ${BENCH_COMMAND}" >&2
echo "Expected one of: cases, latency-sweep, p2p-split-node" >&2
exit 2
;;
esac

View File

@@ -3,6 +3,22 @@ Since 19th July, 2025 (beta1 in 0.25.0-beta1, 13th July, 2025)
The head note of 0.25 is now in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md). Because 0.25 got a lot of updates, thankfully, compatibility is kept and we do not need breaking changes! In other words, when get enough stabled. The next version will be v1.0.0. Even though it my hope.
## 0.25.80
7th July, 2026
### Fixed
- Improved Markdown conflict auto-merge so that non-overlapping edits are merged while overlapping delete-and-edit cases remain visible for manual resolution (#993).
- Behaviour change:
- When one side deletes an unchanged line and the other side edits a different region, the deleted line is no longer reintroduced into the merged result.
- When one side deletes a line and the other side modifies that same line, the conflict is preserved instead of silently choosing one side.
- Fixed an issue where applying a newer database entry to storage could incorrectly preserve an older local file as a conflict (#994).
- Behaviour change:
- Local storage is preserved as a conflict when it may contain unsynchronised changes that are not represented in the revision history. A newer incoming text entry is applied without creating a conflict only when it clearly extends the existing local text.
- Fixed an issue where choosing Disable and then Overwrite in Hidden File Sync could silently skip hidden files, because the overwrite setup ran while hidden file synchronisation was still disabled (#989, PR #992).
- Hidden File Sync is now re-enabled before the Fetch, Overwrite, or Merge initialisation runs, instead of after it completes. If that initialisation fails, the setting may remain enabled.
## 0.25.79
29th June, 2026
@@ -96,41 +112,5 @@ Also, this update is a very large one, even if we had a lot of time, and we had
- Some dependencies have been updated.
- Now we check the compatibility with iOS 15 in the CI tests to ensure the plugin continues to work on older iOS versions even after we upgrade some dependencies.
## 0.25.74
8th June, 2026
### Fixed
- Fixed an issue where disabling hidden file synchronisation did not take effect, allowing non-target hidden files to continue to be processed and synchronised by replication or boot-sequence scan (#941).
- Prevented the automatic merging of conflicted revisions when one of the revisions has been deleted, which was causing deleted files to reappear (#911).
- The startup sequence now saves the state more effectively (Thank you so much for @bmcyver)!
## Only CLI
8th June, 2026
I should also consider the version numbering for the CLI...
### Improved
- Added new remote database management commands: `remote-status`, `unlock-remote`, `lock-remote`, and `mark-resolved`.
- --vault option is now available for daemon and mirror commands! (Thank you so much for @starskyzheng)!
- Decoupled the database directory path from the actual vault directory path using the `--vault` (or `-V`) option.
### Fixed (preventive)
- Validated that the specified vault path exists and is indeed a directory before starting the CLI.
- Integrated path resolution and validations for one-off commands (such as `'push'`, `'pull'`, `'cat'`, `'rm'`, `'info'`, and `'resolve'`) against the decoupled vault path instead of the database path.
## 0.25.73
4th June, 2026
### Fixed
- Adjust CouchDB's database name checking to its specification (#926).
- `Reset Syncronisation on This Device` for minio and P2P is now working properly.
Full notes are in
[updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md).

View File

@@ -4,6 +4,64 @@ Since 19th July, 2025 (beta1 in 0.25.0-beta1, 13th July, 2025)
The head note of 0.25 is now in [updates_old.md](https://github.com/vrtmrz/obsidian-livesync/blob/main/updates_old.md). Because 0.25 got a lot of updates, thankfully, compatibility is kept and we do not need breaking changes! In other words, when get enough stabled. The next version will be v1.0.0. Even though it my hope.
## 0.25.80
7th July, 2026
### Fixed
- Improved Markdown conflict auto-merge so that non-overlapping edits are merged while overlapping delete-and-edit cases remain visible for manual resolution (#993).
- Behaviour change:
- When one side deletes an unchanged line and the other side edits a different region, the deleted line is no longer reintroduced into the merged result.
- When one side deletes a line and the other side modifies that same line, the conflict is preserved instead of silently choosing one side.
- Fixed an issue where applying a newer database entry to storage could incorrectly preserve an older local file as a conflict (#994).
- Behaviour change:
- Local storage is preserved as a conflict when it may contain unsynchronised changes that are not represented in the revision history. A newer incoming text entry is applied without creating a conflict only when it clearly extends the existing local text.
- Fixed an issue where choosing Disable and then Overwrite in Hidden File Sync could silently skip hidden files, because the overwrite setup ran while hidden file synchronisation was still disabled (#989, PR #992).
- Hidden File Sync is now re-enabled before the Fetch, Overwrite, or Merge initialisation runs, instead of after it completes. If that initialisation fails, the setting may remain enabled.
## 0.25.79
29th June, 2026
### Fixed
- Fast Fetch now retries transient stream interruptions and resumes from the latest persisted checkpoint, instead of starting over after ordinary network or platform interruptions (#977, PR #978; commonlib PR #59). Thank you so much for @apple-ouyang for the fix!
- Simple Fetch now remembers the selected setup choices while an interrupted Fetch All operation is still pending, so users are not asked the same questions again on retry (#977, PR #978). Thank you so much for @apple-ouyang for the fix!
- No longer hidden storage events, such as `.git` paths, reach the normal target-file filter when internal file synchronisation is disabled. This avoids noisy non-target logs before those files are skipped (commonlib PR #60). Thank you so much for @apple-ouyang for the fix!
- Fixed an issue where a file deleted from storage could be resurrected by the offline scanner because the database tombstone was not written when the storage file was already gone (commonlib PR #56). Thank you so much for @cosmic-fire-eng for the fix!
### Improved
- Local database maintenance commands now ask before applying the required chunk settings, and can apply those prerequisites before continuing (#980, PR #981). Thank you so much for @apple-ouyang for the improvement!
- Improved CouchDB replication event handling by using the new `StreamInbox` helper from `octagonal-wheels` (commonlib PR #62).
### Documentation
- Added `nginx` to the setup documentation table of contents (PR #976). Thank you so much for @kiraventom for the improvement!
### Miscellaneous
- Updated `octagonal-wheels` to `0.1.47` across the plug-in and workspace packages to use the newly published helper modules.
## 0.25.78
23rd June, 2026
### Fixed
- No longer fast synchronisation (a.k.a. Fast Fetch) causes a rewind and re-fetch of the entire database when some errors occur during the process (#972, PR #973). Thank you so much for @apple-ouyang for the fix!
### Improved
- Overhauled the Object Storage (e.g., MinIO and S3) replication engine ('Journal Replicator 2nd Edition').
- It now leverages the standard Web Streams API for a resilient, backpressure-aware architecture, reducing memory footprints/temporary storage usage on large vaults.
- Decoupled the physical storage logic to make it easier to add new storage backends in the future.
- Stricter compliance with CouchDB's replication protocol (proper `_revisions` transfers with `new_edits: false`) when using Object Storage.
### Testing
- Added comprehensive unit tests for the new `JournalSyncCore` engine, covering streams, backpressure, and `new_edits: false` validation.
- Improved integration test workflows in the CI pipeline to run MinIO tests automatically using standard environment variables.
## 0.25.77
19th June, 2026