Compare commits

...

31 Commits

Author SHA1 Message Date
SoftFever
8c050894db Ship one preset cache per vendor in place of the profile JSONs
Each vendor's system presets serialize into a single <vendor>.opc built at
package time, and a shipped build carries that file alone — the profile JSON
and its sub-file tree are pruned. The vendor loader, the setup wizard's profile
list and the resource installer all read a vendor through its cache, falling
back to parsing whenever one is absent, stale or unreadable, so the cache stays
an optimization and never a source of truth. Caches hold presets in source form
and resolve inheritance at load, through the same code the JSON path uses.
2026-08-05 21:16:41 +08:00
ExPikaPaka
55fb4703c9 Remove not used files 2026-08-05 21:15:12 +08:00
ExPikaPaka
b25d998fc5 Ignore cache for future 2026-08-05 21:15:11 +08:00
ExPikaPaka
bc6f3aa7f7 Transfer all Preset fields from cache via move assignmet
apply_vendor_preset_group was copying fields manually and missed
bundle_id, user_id, base_id, sync_info, updated_time, key_values,
ini_str. Replace field-by-field copy with move assignment of the
fully-deserialized Preset, then restore the vendor pointer which
is excluded from serialization.
2026-08-05 21:15:11 +08:00
ExPikaPaka
5bf0e09781 Rvert from per-verndor to single cache file
Replace N per-vendor .cache files with a single system_presets.cache
that holds all vendors and presets in one serialized blob.

Cache load is now all-or-nothing: on hit all vendors are applied from
the bundle (sub-second); on miss all vendors are parsed from JSON and
a fresh bundle is written to the user cache dir.

Invalidation is driven by bundle_key - a sorted concatenation of all
vendor JSON version strings. Any vendor update invalidates the whole
cache and triggers re-parse on next launch.

Guide wizard (WebGuideDialog) loads the bundled cache into a plain
PresetBundle instead of a separate VendorGuideData struct, removing
the duplicate data model.

generate_system_cache simplified from a per-vendor loop to a single
save_system_presets_cache() call producing one output file.
2026-08-05 21:15:11 +08:00
ExPikaPaka
3fea10a711 Merge branch 'main' into feature/cache_profiles_and_optimize_loading_speed 2026-08-05 21:15:11 +08:00
ExPikaPaka
0df7d9e661 Add a bit more tests 2026-08-05 21:15:11 +08:00
ExPikaPaka
603b1eaa87 Add tests for Cache system 2026-08-05 21:15:10 +08:00
ExPikaPaka
cae9c6dfa6 Minimize field duplication by moving Cache thing into PresetBundle 2026-08-05 21:14:56 +08:00
ExPikaPaka
1f35b28a50 Serealize all value fields for Preset class to minimize regression later 2026-08-05 21:14:56 +08:00
ExPikaPaka
68e7810c73 Update check for stale cache 2026-08-05 21:14:56 +08:00
ExPikaPaka
2ac64f7171 Revert json cache back 2026-08-05 21:14:56 +08:00
ExPikaPaka
c0cfcbbc64 Fix build for windows arm64 2026-08-05 21:14:55 +08:00
ExPikaPaka
3c6d104c71 Remove leftover cache file 2026-08-05 21:14:55 +08:00
ExPikaPaka
d85b555748 Skip invalid vendors 2026-08-05 21:14:55 +08:00
ExPikaPaka
ff92efa91f Remove BOM added by VSC 2026-08-05 21:14:55 +08:00
ExPikaPaka
89b66daeca Use get_vendor_cache_key() to match cache keys written by the app 2026-08-05 21:14:55 +08:00
ExPikaPaka
1f29ef3adc Fix use-after-free in CallAfter lambda; replace raw thread pointer with unique_ptr 2026-08-05 21:14:55 +08:00
ExPikaPaka
fe216a4ded Remove CachedPrinterModel/VendorProfile/Preset mirror structs from VendorCache 2026-08-05 21:14:55 +08:00
ExPikaPaka
8ea0b121e7 Add cereal serialize() to VendorProfile, PrinterModel, Preset, and Semver 2026-08-05 21:14:55 +08:00
ExPikaPaka
1d19e39625 Simplify code a bit more 2026-08-05 21:14:55 +08:00
ExPikaPaka
273d325066 Simplify code by mergin it in PresetBundle 2026-08-05 21:14:55 +08:00
ExPikaPaka
5ad7a3eb5d Generate cache per vendor 2026-08-05 21:14:55 +08:00
ExPikaPaka
c12e348ce3 Add inspecting tool and fix CI cache generation 2026-08-05 21:14:55 +08:00
ExPikaPaka
b34923fc7c Add cache to GuideDialog as previos version didn't work as expected 2026-08-05 21:14:55 +08:00
ExPikaPaka
a1eba4b2c8 Handle corrupted files 2026-08-05 21:14:55 +08:00
ExPikaPaka
6b45fa2c97 Add partial cache generation when only one of the vendros is changed to speed up recalculation time 2026-08-05 21:14:55 +08:00
ExPikaPaka
6b3e4f18e1 Add CI\CD step to prepare cache file in ahead of time so user does not need to wait 2026-08-05 21:14:55 +08:00
ExPikaPaka
25b1577c45 Integrate caching into WebGuideDialog which speeds up time of SetupWizzard and PrinterSelection dialog 2026-08-05 21:13:48 +08:00
ExPikaPaka
69c1f6880c Removing user\bundle serialization and keeping it only for system presets 2026-08-05 21:13:48 +08:00
ExPikaPaka
86bedc1e37 Add caching system for presets 2026-08-05 21:13:48 +08:00
28 changed files with 3359 additions and 609 deletions

5
.gitattributes vendored
View File

@@ -1,2 +1,7 @@
# Set the default behavior, in case people don't have core.autocrlf set.
* text=auto
# Shell scripts are run by Git Bash on Windows CI, which cannot read a script
# with CRLF line endings: it fails on the first line. Windows checkouts default
# to core.autocrlf=true, so keep these LF whatever the platform.
*.sh text eol=lf

View File

@@ -162,6 +162,14 @@ jobs:
retention-days: 5
if-no-files-found: error
- name: Build system preset cache (macOS)
if: runner.os == 'macOS' && !inputs.macos-combine-only
working-directory: ${{ github.workspace }}
shell: bash
# The bundle was already packed from resources/, so the caches have to be
# installed into it here; the source tree keeps its JSONs for later jobs.
run: ./scripts/build_preset_cache.sh -b build/${{ inputs.arch }} build/${{ inputs.arch }}/OrcaSlicer/OrcaSlicer.app/Contents/Resources/profiles
- name: Pack macOS app bundle ${{ inputs.arch }}
if: runner.os == 'macOS' && !inputs.macos-combine-only
working-directory: ${{ github.workspace }}
@@ -390,6 +398,13 @@ jobs:
if ($arch -eq "arm64") { .\build_release_vs.bat slicer arm64 tests } else { .\build_release_vs.bat slicer tests }
shell: pwsh
- name: Build system preset cache (Windows)
if: runner.os == 'Windows'
shell: cmd
# Shipped into both the already-installed tree (portable zip, MSIX) and
# the checkout cpack re-installs from when it builds the NSIS installer.
run: scripts\build_preset_cache.bat "%BUILD_DIR%" "resources\profiles" "%BUILD_DIR%\OrcaSlicer\resources\profiles"
- name: Pack unit tests Win
if: runner.os == 'Windows'
working-directory: ${{ github.workspace }}
@@ -539,6 +554,20 @@ jobs:
retention-days: 5
if-no-files-found: error
- name: Build system preset cache (Linux)
if: runner.os == 'Linux'
shell: bash
run: |
# Both were packed from resources/ before the caches existed, so the
# AppImage is unpacked first and the caches shipped into it and into
# the package tree; the source tree keeps its JSONs for later steps.
appimage=$(find build -maxdepth 1 -name "OrcaSlicer_Linux_AppImage*.AppImage" | head -1)
chmod +x "$appimage"
"$appimage" --appimage-extract
./scripts/build_preset_cache.sh -b build build/package/resources/profiles squashfs-root/resources/profiles
appimagetool=$(find build -name "appimagetool.AppImage" | head -1)
ARCH=$(uname -m) "$appimagetool" --appimage-extract-and-run squashfs-root "$appimage"
rm -rf squashfs-root
# Ship the freshly-built validator so slice_check_linux (build_all.yml)
# can slice-sweep the shipped profiles with this PR's engine. Taken from
# the aarch64 leg so the sweep also exercises the arm build; x86_64 on

1
.gitignore vendored
View File

@@ -49,3 +49,4 @@ internal_docs/
# Python bytecode
__pycache__/
*.pyc
*.opc

View File

@@ -567,6 +567,8 @@ if [[ -n "${BUILD_ORCA}" ]] || [[ -n "${BUILD_TESTS}" ]] ; then
print_and_run cmake --build $BUILD_DIR --config "${BUILD_CONFIG}" --target OrcaSlicer
echo "Building OrcaSlicer_profile_validator .."
print_and_run cmake --build $BUILD_DIR --config "${BUILD_CONFIG}" --target OrcaSlicer_profile_validator
echo "Building generate_system_cache ..."
print_and_run cmake --build $BUILD_DIR --config "${BUILD_CONFIG}" --target generate_system_cache
./scripts/run_gettext.sh
fi
if [[ -n "${BUILD_TESTS}" ]] ; then

288
docs/HLSD/preset-cache.md Normal file
View File

@@ -0,0 +1,288 @@
# System Preset Cache — High Level Design
## Why it exists
OrcaSlicer ships tens of thousands of system preset JSON files. Every launch used to
parse all of them: read each vendor profile, walk its machine, process and filament
sub-files, resolve inheritance, and build the preset collections from scratch. That
parse dominated startup, and it produced the same result every time, because system
presets only change when the app is updated or a profile update is installed.
The preset cache replaces that parse with a read. Each vendor's presets are serialized
once — at build time, in CI — into a single binary file the app reads in one pass. The
read replaces the file walk and the JSON parsing, which is where the time went;
resolving inheritance and registering the presets still runs at load, through the same
code the JSON path uses, so the result is the parse's result without the parse.
The cache is **only ever an optimization**. Every rule below exists to guarantee that a
cache is either provably equivalent to parsing the JSONs, or rejected. There is no
"mostly right" cache.
## The unit is one vendor
A cache covers exactly one vendor. `BBL.opc` sits beside `BBL.json` and holds
everything `BBL.json` and the `BBL/` sub-file tree would have produced.
Per-vendor granularity is what makes the system practical:
- A vendor whose profile is bumped invalidates only its own cache. The other 60-odd
vendors keep theirs — even when the bumped vendor is the shared Orca filament
library everyone else inherits from.
- The setup wizard, which loads vendors one at a time, gets the same speedup as
startup without a second code path.
- A vendor with no cache, or a broken one, costs only that vendor a parse.
A cache holds *system* presets only. User presets, project settings and modified
presets are never serialized — they have their own storage and their own lifecycle.
## Where the files live
| Location | Contents on a shipped build | Role |
|---|---|---|
| `resources/profiles/` | `<vendor>.opc` alone — the profile and its preset JSONs both pruned | What the app ships with; the fallback everything falls back to |
| `<data_dir>/system/` | `<vendor>.opc` alone, or `<vendor>.json` + `<vendor>/` after an update | What the user has installed |
| `<data_dir>/system/` (dev build) | `<vendor>.json` + `<vendor>/` + `<vendor>.opc` written at runtime | A developer tree caches as it parses |
Two forms of the same vendor therefore exist, and the system's central rule is that
**a vendor's cache is the whole of it**. Where a cache ships or is installed, no profile
and no preset JSONs sit beside it: the cache carries the presets, the vendor profile,
and the version stamp that says which release it came from. A vendor is "installed" if
either form is present, and its installed version is read from whichever form is there.
What stays beside the caches in `resources/profiles/` is everything that is not a
preset: each vendor's directory of printer thumbnails, cover images, bed models and
hotend meshes, which are read from disk by path and were never part of the cache. Files
that are not vendors at all, `blacklist.json` chief among them, are untouched.
The alternative — shipping both and treating the cache as a sidecar — was rejected. It
doubles the installed size, and it creates a class of bug where the two disagree and
the app's behavior depends on which one a given code path happened to read.
## What a cache file is
A fixed-size header followed by one binary stream.
The header carries a magic number, the cache format version, the payload size and a
CRC32 of the payload. It exists so that a truncated download, a half-written file or a
file from an entirely different program is rejected in microseconds, before anything
tries to interpret it.
The payload opens with the stamps that decide whether the cache may be used at all —
format version, schema fingerprint, vendor name, vendor version — and then the
vendor's data: its vendor profile, three lists of preset entries (process, filament,
machine), and the count of errors the original parse hit.
Each entry is one preset **in source form**: what its JSON sub-file states and nothing
that resolving it derives — the preset's own config diff, the name of the preset it
inherits, and the parse metadata (name, sub-path, description, instantiation, setting
and filament ids, renames). Non-instantiated base presets are stored too; the children
that inherit from them cannot resolve without them.
Three deliberate choices in the layout:
- **Stamps come first**, so the question "what version is this vendor installed at?"
can be answered by reading the first kilobyte. The updater asks that question for
every vendor on every launch; reading tens of megabytes to answer it would give back
the startup time the cache saved.
- **Nothing inherited is baked in.** A filament preset that inherits from the shared
library is stored as its own diff plus its parent's name, and the parent is looked up
when the entry is installed, against whatever library is loaded then. A cache
therefore carries no other vendor's values, and no other vendor's update — the
library's included — can make it stale.
- **Nothing derived is stored.** Default presets, flattened configs, aliases and
lookup maps are all reconstructed at load by the same code the JSON path runs, and
state that path never fills (obsolete-preset lists) is not stored either. This keeps
the cache a record of the vendor's data, not a memory image of the program's state.
## When a cache may be used
A cache is accepted only if every gate below passes. Any failure means "parse the
JSONs instead" — never a hard error, never a partial load.
**1. Integrity.** Magic number, plausible size, CRC32 over the payload.
**2. Cache format version.** A single integer bumped by hand whenever the binary layout
changes in a way nothing else would catch: reordering or retyping a hand-written
serialized field, or changing what the cache's own stamps mean.
**3. Schema fingerprint.** A checksum over the app version and the entire print-config
option schema — every option's key, type, wire ordinal and enum values. This is the
gate that makes the cache safe across development: adding a config option, changing its
type, or reordering the enum values of an existing one all change the fingerprint, so
caches from before the change are rejected without anyone having to remember to bump
anything. It also means a cache never crosses app versions.
**4. Vendor identity and version.** The cache names the vendor it holds and the profile
version it was built from. It is accepted only if that version is at least as new as
the profile now on disk. Where no profile sits beside the cache — the shipped,
cache-only form — the comparison is skipped, because nothing on disk can be newer than
a cache that is the installation.
**5. Every entry installs.** Entries are installed as they are read, and an entry that
cannot be — typically one that inherits a parent the currently loaded filament library
no longer provides — rejects the whole cache, never just the entry. A partial vendor is
not a vendor.
There is deliberately no stamp for the shared filament library. A cache stores its
filaments' inheritance by name and resolves it at load, so a library update changes
what a cache load *produces*, never whether the cache is *valid* — the same file yields
the updated result. This matters most on a shipped build, where a vendor is its cache
and nothing else: a profile update that delivered only the library would otherwise have
stranded every other vendor with a cache it invalidated and no JSONs to fall back on.
A vendor profile with no parsable version is never cached and never served from a
cache. There would be no way to tell later whether the cache had gone stale, and a
cache nothing can invalidate is worse than no cache.
## How a vendor is loaded
Vendors load in a fixed order, because filament inheritance crosses exactly one
boundary: any vendor's filament may inherit from the shared Orca filament library,
and nothing else reaches across vendors. The library therefore goes first, alone;
every other vendor follows in parallel, resolving against it; and the results are
merged in a stable order:
```mermaid
flowchart LR
lib["1 · OrcaFilamentLibrary<br/>loaded first, synchronously"] --> par["2 · every other vendor in parallel,<br/>each into its own bundle, filaments<br/>resolving against the loaded library"] --> merge["3 · bundles merged into one,<br/>sequentially, in stable vendor order"]
```
Whether a vendor comes from its cache or from a parse changes nothing in that
order — both produce the same bundle, so cached and parsed vendors mix freely in
one startup. Each vendor load — startup's and the setup wizard's alike — tries,
in order:
1. The cache in the directory it was asked to load from — normally `<data_dir>/system/`.
2. The shipped cache in `resources/profiles/` — judged against the same installed
profile, so it cannot resurrect a version an update has superseded.
3. Parsing the JSONs — from the data directory if the profile is installed there, and
from `resources/profiles/` otherwise, which on a shipped build only has JSONs for a
vendor that has no cache.
The same decision drawn out — "the gates" are the five acceptance checks above:
```mermaid
flowchart TB
start["load vendor V from a directory dir<br/>— normally &lt;data_dir&gt;/system/"]
start --> stamp["installed version = version of dir/V.json<br/>— or ∞ with no profile there,<br/>the cache then being the installation"]
stamp --> g1{"dir/V.opc<br/>passes all five gates?"}
g1 -- "yes" --> hit(["served from the<br/>installed cache"])
g1 -- "no" --> g2{"resources/profiles/V.opc passes them,<br/>against the same installed version?"}
g2 -- "yes" --> hit2(["served from the<br/>shipped cache"])
g2 -- "no" --> src{"dir/V.json<br/>present?"}
src -- "yes" --> pd["parse the JSONs in dir"]
src -- "no" --> pr["parse the JSONs in<br/>resources/profiles/"]
pd --> ver{"profile version<br/>parsable?"}
pr --> ver
ver -- "yes" --> save(["loaded; dir/V.opc written back —<br/>the next load takes the top path"])
ver -- "no" --> raw(["loaded, never cached"])
```
Serving from a cache is not a memory-image restore. The entries are deserialized and
then installed one by one — inheritance resolved against the presets installed before
them and the currently loaded filament library, configs flattened onto the collection
defaults, validated and registered — by the same function the JSON path calls straight
after parsing a sub-file. The two paths share everything below the parse, which is what
makes a cache-loaded bundle indistinguishable from a JSON-loaded one by construction
rather than by test coverage. Installation also rebuilds each preset's file path from
the local data directory, so a shipped cache never carries the generating machine's
paths.
The second tier is what makes app upgrades work. After an upgrade, a cache the previous
version installed fails the fingerprint gate; the new build's own shipped cache answers
instead, and the user never sees a parse. The stale installed file is simply ignored
until the next profile update overwrites it.
If a parse does happen and the vendor's profile carries a version, the app writes the
cache back beside where it looked for the vendor. That is how a developer build warms
itself up on second launch, and how a vendor delivered by a profile update becomes
cached without waiting for the next release.
## How a vendor is installed
Installing copies from `resources/profiles/` into `<data_dir>/system/`. A shipped build
offers only a cache and a source tree only JSONs, but a partially-generated tree can
have both, at different versions, so the installer picks the form that ships at the
**newer version** and installs only that one:
- Cache newer or equal, and readable → copy the `.opc`, and delete any profile and
vendor directory a previous install left behind, so nothing can shadow it.
- Profile newer, or the cache unreadable or absent → copy the profile and the vendor's
preset JSONs exactly as the app did before caches existed, and delete any stale `.opc`.
The result is that only one form of a vendor is ever present, and it is the newest one
the build has. This matters most for the update check, which compares what is installed
against what installing *would* lay down: if those two disagreed about which form
counts, a vendor could reinstall on every launch forever, or silently never update.
Profile updates delivered over the air always arrive as JSONs, and they win — an
updated vendor's real profile lands in the data directory, the shipped cache is older
and gets rejected, and the vendor is parsed and re-cached. An update that touches only
the filament library needs nothing more: every other vendor's cache stays valid and
simply resolves against the new library on its next load.
## How the caches are produced
Cache generation is a build step, not something a user ever runs.
One script per platform does the whole job, and CI calls it once on each. It builds a
small dev-utility that loads a profiles directory exactly as the app would, with cache
writing enabled, dropping a `<vendor>.opc` beside every vendor profile it parses; then
it copies those caches into each packaged application it was pointed at and deletes
every preset JSON they replace — the vendor's own profile included. Only a vendor that
actually has a cache is pruned, so a vendor the generator skipped keeps its JSONs and is
simply parsed at startup.
Because the schema fingerprint includes the app version, caches must be generated by
the same build that ships them. Generation runs after the build, in the same job.
## Behavior when things go wrong
The system is designed so that no cache problem is fatal:
- **Corrupt, truncated or foreign file** — rejected at the header, vendor parsed.
- **Cache from another app version or schema** — rejected at the fingerprint, vendor
parsed or served from the shipped cache.
- **Stale cache** — rejected on the vendor version stamp, vendor parsed and re-cached.
- **Failure part-way through loading** — a deserialization error, or any entry that
fails to install — rejects the whole cache, and the bundle is reset to a clean state
before falling back, so a half-loaded cache can never leak into the parsed result.
- **A vendor that can be neither read nor parsed** — logged, and left out. The setup
wizard drops that vendor from its list and opens with the rest; startup records the
error alongside the vendors that did load. One broken vendor never takes the app down.
The one genuine limit: on a shipped build a vendor is its cache and nothing else, so a
rejected cache has nothing to fall back to for that vendor. This is by design — the
alternative is shipping every preset twice — and it is why the acceptance gates are
conservative and why CI generates the caches with the same build that ships them. The
recovery path is a profile update, which delivers real JSONs.
It also means nothing may quietly assume a `<vendor>.json` exists. Discovery, version
checks and the update decision all read whichever form is present, and a code path that
enumerates only `*.json` will find no vendors at all in a packaged build.
## Maintenance rules
- **Adding or changing a config option** needs nothing. The fingerprint covers it.
- **Changing a hand-written `serialize()`** — `PresetBundle::CachedPreset`,
`VendorProfile` or its nested types — or the cache's own layout or stamps requires
bumping the cache format version by hand. Nothing else is serialized by hand; the
config payload is covered by the fingerprint.
- **Bumping a vendor profile's version** invalidates that vendor's cache and nothing
else — the filament library's included. Other vendors' caches resolve against the
new library the next time they load.
- **Caches are never committed.** They are build artifacts, generated per build,
ignored by git.
## Where this lives in the tree
| Area | Files |
|---|---|
| Cache format, entry serialization, read/write, load and save | `src/libslic3r/PresetBundle.{hpp,cpp}` |
| Vendor profile serialization | `src/libslic3r/Preset.hpp` |
| Vendor discovery, installed/shipped versions, installation | `src/libslic3r/PresetBundle.cpp` |
| Update and reinstall decisions | `src/slic3r/Utils/PresetUpdater.cpp` |
| Setup wizard and printer-selection dialog | `src/slic3r/GUI/ConfigWizard.cpp`, `src/slic3r/GUI/WebGuideDialog.cpp` |
| Generator tool | `src/dev-utils/generate_system_cache.cpp` |
| Build and packaging script | `scripts/build_preset_cache.{sh,bat}` |
| Tests | `tests/libslic3r/test_vendor_cache.cpp` |

View File

@@ -0,0 +1,122 @@
@echo off
rem Build the per-vendor system preset caches (one <vendor>.opc per vendor) by
rem running the generate_system_cache.exe dev tool against a profiles directory,
rem and make every profiles directory named on the command line ship-ready:
rem install the caches into it and delete the preset JSONs they replace, so a
rem build ships one copy of its presets instead of two.
rem
rem scripts\build_preset_cache.bat [build_dir] [target_dir ...]
rem
rem build_dir defaults to "build"
rem target_dir profiles directories to ship into. Caches are generated into
rem the source tree's resources\profiles, which is what every
rem packaging step copies from; a target may be that same
rem directory, which then only gets pruned.
rem
rem Shipping deletes, so it is a CI packaging step. A vendor's own <vendor>.json
rem goes along with its preset JSONs: the cache carries the vendor profile and
rem the version it was built at, so discovery, version checks and installing all
rem read it there. Only a vendor that has a cache is pruned, so non-vendor JSONs
rem (blacklist.json) are left alone, as are the vendor directories themselves -
rem thumbnails, covers and bed models still live there.
rem
rem set CONFIG=<cfg> to pin the build config for multi-config generators
rem (default: the config of the tool already in the build tree, else Release)
setlocal enabledelayedexpansion
set "REPO_ROOT=%~dp0.."
set "BUILD_DIR=%~1"
if "%BUILD_DIR%"=="" set "BUILD_DIR=build"
if not exist "%BUILD_DIR%\" (
echo ERROR: build tree not found: %BUILD_DIR% 1>&2
exit /b 1
)
if not "%~1"=="" shift
rem Newest match wins: a stale binary silently produces a stale cache layout.
call :find_tool
if not defined CONFIG (
for %%c in (Debug Release RelWithDebInfo MinSizeRel) do (
echo !TOOL! | findstr /i "\\%%c\\" >nul && set "CONFIG=%%c"
)
)
if not defined CONFIG set "CONFIG=Release"
echo Building generate_system_cache in %BUILD_DIR% (%CONFIG%)
cmake --build "%BUILD_DIR%" --config %CONFIG% --target generate_system_cache
if errorlevel 1 (
echo ERROR: could not build generate_system_cache - configure the build tree with -DORCA_TOOLS=ON: 1>&2
echo cmake -S "%REPO_ROOT%" -B "%BUILD_DIR%" -DORCA_TOOLS=ON 1>&2
exit /b 1
)
call :find_tool
if not defined TOOL (
echo ERROR: generate_system_cache.exe not found under %BUILD_DIR% - build with -DORCA_TOOLS=ON 1>&2
exit /b 1
)
set "PROFILES=%REPO_ROOT%\resources\profiles"
if not exist "%PROFILES%\" (
echo ERROR: profiles directory not found: %PROFILES% 1>&2
exit /b 1
)
for %%d in ("%PROFILES%") do set "PROFILES=%%~fd"
rem Add the slicer's runtime DLL directory to PATH so generate_system_cache.exe
rem can resolve its dependencies (TKernel.dll etc.) without a full install step.
set "DLL_DIR="
for /f "delims=" %%f in ('dir /s /b "%BUILD_DIR%\TKernel.dll" 2^>nul') do (
if not defined DLL_DIR set "DLL_DIR=%%~dpf"
)
if defined DLL_DIR set "PATH=%DLL_DIR%;%PATH%"
echo Generating per-vendor preset caches in %PROFILES%
rem Start clean so vendors that went away - and caches written by older tool
rem versions - don't linger next to the freshly generated ones.
del /q "%PROFILES%\*.opc" 2>nul
del /q "%PROFILES%\*.cache" 2>nul
"%TOOL%" --path "%PROFILES%" --log_level 2
if errorlevel 1 exit /b %errorlevel%
:next_target
if "%~1"=="" exit /b 0
call :ship "%~1"
if errorlevel 1 exit /b 1
shift
goto :next_target
:ship
set "TARGET=%~1"
if not exist "%TARGET%\" (
echo ERROR: profiles directory not found: %TARGET% 1>&2
exit /b 1
)
for %%d in ("%TARGET%") do set "TARGET=%%~fd"
if /i not "%TARGET%"=="%PROFILES%" copy /y "%PROFILES%\*.opc" "%TARGET%\" >nul
set /a SHIPPED=0
set /a PRUNED=0
for %%c in ("%PROFILES%\*.opc") do (
set /a SHIPPED+=1
set "VENDOR=%%~nc"
if exist "%TARGET%\!VENDOR!.json" (
del /q "%TARGET%\!VENDOR!.json"
set /a PRUNED+=1
)
if exist "%TARGET%\!VENDOR!\" (
for /f %%n in ('dir /s /b "%TARGET%\!VENDOR!\*.json" 2^>nul ^| find /c /v ""') do set /a PRUNED+=%%n
del /s /q "%TARGET%\!VENDOR!\*.json" >nul 2>&1
rem Deepest first, so a directory the delete above emptied goes too; rd
rem refuses the ones still holding covers or meshes.
for /f "delims=" %%d in ('dir /s /b /ad "%TARGET%\!VENDOR!" 2^>nul ^| sort /r') do rd "%%d" 2>nul
)
)
echo %TARGET%: !SHIPPED! caches, dropped !PRUNED! preset JSONs
exit /b 0
:find_tool
set "TOOL="
for /f "delims=" %%f in ('dir /s /b /o-d "%BUILD_DIR%\generate_system_cache.exe" 2^>nul') do (
if not defined TOOL set "TOOL=%%f"
)
exit /b 0

143
scripts/build_preset_cache.sh Executable file
View File

@@ -0,0 +1,143 @@
#!/usr/bin/env bash
# Build the per-vendor system preset caches (one <vendor>.opc per vendor) by
# running the generate_system_cache dev tool against a profiles directory, and
# make every profiles directory named on the command line ship-ready: install
# the caches into it and delete the preset JSONs they replace, so a build ships
# one copy of its presets instead of two.
#
# ./scripts/build_preset_cache.sh # caches into resources/profiles
# ./scripts/build_preset_cache.sh -b build/arm64 # search this build tree for the tool
# ./scripts/build_preset_cache.sh <dir> [<dir> ...] # and ship into these profiles dirs
#
# Caches are generated into the source tree's resources/profiles, which is what
# every packaging step copies from. Shipping deletes, so it is a CI packaging
# step: pass packaged output directories, or the checkout of a build that is
# about to be packaged from it.
#
# A vendor's own <vendor>.json goes along with its preset JSONs: the cache
# carries the vendor profile and the version it was built at, so discovery,
# version checks and installing all read it there. A shipped vendor is its cache
# and nothing else. Only a vendor that has a cache is pruned, so an ungenerated
# vendor keeps its JSONs and is simply parsed at startup; non-vendor JSONs
# (blacklist.json) are left alone, as are the vendor directories themselves —
# thumbnails, covers and bed models still live there.
#
# -b <dir> build tree holding the tool
# (default: build/arm64, build/x86_64, or build — first that exists)
# -p <dir> profiles directory to generate caches into
# (default: <repo>/resources/profiles)
# -c <cfg> build config for multi-config generators
# (default: the config of the tool already in the build tree, else
# the build tree's CMAKE_BUILD_TYPE)
# -n skip the rebuild and run the tool already in the build tree
# -l <level> tool log level (default: 2)
set -euo pipefail
repo_root="$(cd "$(dirname "$0")/.." && pwd -P)"
build_dir=""
profiles_dir=""
config=""
build_tool=1
log_level=2
while getopts "b:p:c:l:nh" opt; do
case $opt in
b) build_dir="$OPTARG" ;;
p) profiles_dir="$OPTARG" ;;
c) config="$OPTARG" ;;
n) build_tool=0 ;;
l) log_level="$OPTARG" ;;
h) sed -n '2,${/^#/!q;p;}' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
*) exit 1 ;;
esac
done
shift $((OPTIND - 1))
if [ -z "$build_dir" ]; then
for candidate in "$repo_root/build/arm64" "$repo_root/build/x86_64" "$repo_root/build"; do
if [ -d "$candidate" ]; then build_dir="$candidate"; break; fi
done
fi
if [ -z "$build_dir" ] || [ ! -d "$build_dir" ]; then
echo "ERROR: build tree not found (pass -b <build_dir>)" >&2
exit 1
fi
# Newest match wins: multi-config trees keep one binary per config, and a stale
# one silently produces a stale cache layout.
find_tool() {
local best="" f
while IFS= read -r f; do
[ -n "$f" ] || continue
if [ -z "$best" ] || [ "$f" -nt "$best" ]; then best="$f"; fi
done < <(find "$build_dir" -name generate_system_cache -type f 2>/dev/null)
printf '%s' "$best"
}
tool=$(find_tool)
if [ -z "$config" ]; then
case "$tool" in
*/Debug/*) config=Debug ;;
*/Release/*) config=Release ;;
*/RelWithDebInfo/*) config=RelWithDebInfo ;;
*/MinSizeRel/*) config=MinSizeRel ;;
*) config=$(sed -n 's/^CMAKE_BUILD_TYPE:[A-Z]*=\(.\+\)$/\1/p' "$build_dir/CMakeCache.txt" 2>/dev/null | head -1 || true) ;;
esac
fi
if [ "$build_tool" = 1 ]; then
echo "Building generate_system_cache in $build_dir${config:+ ($config)}"
build_args=(--build "$build_dir" --target generate_system_cache)
if [ -n "$config" ]; then build_args+=(--config "$config"); fi
if ! cmake "${build_args[@]}"; then
echo "ERROR: could not build generate_system_cache — configure the build tree with -DORCA_TOOLS=ON:" >&2
echo " cmake -S \"$repo_root\" -B \"$build_dir\" -DORCA_TOOLS=ON" >&2
exit 1
fi
tool=$(find_tool)
fi
if [ -z "$tool" ]; then
echo "ERROR: generate_system_cache not found under $build_dir — build with -DORCA_TOOLS=ON" >&2
exit 1
fi
if [ -z "$profiles_dir" ]; then profiles_dir="$repo_root/resources/profiles"; fi
if [ ! -d "$profiles_dir" ]; then
echo "ERROR: profiles directory not found: $profiles_dir" >&2
exit 1
fi
profiles_dir=$(cd "$profiles_dir" && pwd -P)
# Start clean so vendors that went away — and caches written by older tool
# versions — don't linger next to the freshly generated ones.
echo "Generating per-vendor preset caches in $profiles_dir"
rm -f "$profiles_dir"/*.opc "$profiles_dir"/*.cache
"$tool" --path "$profiles_dir" --log_level "$log_level"
for target in "$@"; do
resolved=$(cd "$target" 2>/dev/null && pwd -P) || {
echo "ERROR: profiles directory not found: $target" >&2
exit 1
}
if [ "$resolved" != "$profiles_dir" ]; then
cp "$profiles_dir"/*.opc "$resolved"/
fi
pruned=0
shipped=0
for cache in "$profiles_dir"/*.opc; do
vendor=$(basename "$cache" .opc)
shipped=$(( shipped + 1 ))
if [ -f "$resolved/$vendor.json" ]; then
rm -f "$resolved/$vendor.json"
pruned=$(( pruned + 1 ))
fi
[ -d "$resolved/$vendor" ] || continue
n=$(find "$resolved/$vendor" -name '*.json' | wc -l)
find "$resolved/$vendor" -name '*.json' -delete
find "$resolved/$vendor" -type d -empty -delete
pruned=$(( pruned + n ))
done
echo "$resolved: $shipped caches, dropped $pruned preset JSONs"
done

View File

@@ -20,6 +20,16 @@ if (SLIC3R_ENC_CHECK)
)
endif()
if (ORCA_TOOLS)
set(_DEV_DEFS -DBOOST_ALL_NO_LIB -DBOOST_USE_WINAPI_VERSION=0x602 -DBOOST_SYSTEM_USE_UTF8)
# generate_system_cache: pre-generates per-vendor <vendor>.opc files under resources/profiles for CI bundling.
add_executable(generate_system_cache generate_system_cache.cpp)
target_link_libraries(generate_system_cache libslic3r boost_headeronly)
target_compile_definitions(generate_system_cache PRIVATE ${_DEV_DEFS})
endif()
# Function that adds source file encoding check to a target
# using the above encoding-check binary

View File

@@ -0,0 +1,84 @@
#include "libslic3r/PresetBundle.hpp"
#include "libslic3r/Preset.hpp"
#include "libslic3r/Utils.hpp"
#include <boost/algorithm/string/predicate.hpp>
#include <boost/filesystem.hpp>
#include <boost/log/trivial.hpp>
#include <boost/program_options.hpp>
#include <iostream>
using namespace Slic3r;
namespace fs = boost::filesystem;
namespace po = boost::program_options;
int main(int argc, char* argv[])
{
po::options_description desc("OrcaSlicer System Cache Generator\nUsage");
// clang-format off
desc.add_options()
("help,h", "Show help")
#ifdef __APPLE__
("path,p", po::value<std::string>()->default_value("../../../../../../../resources/profiles"), "Path to profiles directory")
#else
("path,p", po::value<std::string>()->default_value("../../../resources/profiles"), "Path to profiles directory")
#endif
("log_level,l", po::value<int>()->default_value(2), "Log level (0=trace, 2=info, 4=error)");
// clang-format on
po::variables_map vm;
try {
po::store(po::parse_command_line(argc, argv, desc), vm);
if (vm.count("help")) { std::cout << desc << "\n"; return 0; }
po::notify(vm);
} catch (const po::error& e) {
std::cerr << "Error: " << e.what() << "\n" << desc << "\n";
return 1;
}
const std::string profiles_path = vm["path"].as<std::string>();
const int log_level = vm["log_level"].as<int>();
if (!fs::exists(profiles_path) || !fs::is_directory(profiles_path)) {
std::cerr << "Error: '" << profiles_path << "' is not a valid directory\n";
return 1;
}
set_logging_level(log_level);
set_data_dir(profiles_path);
set_resources_dir(fs::path(profiles_path).parent_path().make_preferred().string());
const fs::path user_dir = fs::path(data_dir()) / PRESET_USER_DIR;
if (!fs::exists(user_dir))
fs::create_directories(user_dir);
AppConfig app_config;
app_config.set("preset_folder", "default");
auto preset_bundle = std::make_unique<PresetBundle>();
preset_bundle->set_is_validation_mode(true);
preset_bundle->set_default_suppressed(true);
preset_bundle->set_generate_vendor_caches(true);
std::cout << "Loading system presets from: " << profiles_path << "\n";
try {
// In validation mode data_dir() is the profiles directory set above, so the
// loader writes each <vendor>.opc next to its <vendor>.json as it parses it.
preset_bundle->load_presets(app_config, ForwardCompatibilitySubstitutionRule::EnableSilent);
} catch (const std::exception& ex) {
std::cerr << "Failed to load presets: " << ex.what() << "\n";
return 1;
}
size_t cache_count = 0;
for (auto& entry : fs::directory_iterator(profiles_path))
if (boost::iends_with(entry.path().string(), ".opc"))
++ cache_count;
if (cache_count == 0) {
std::cerr << "No vendor cache files were generated under " << profiles_path << "\n";
return 1;
}
std::cout << "Generated " << cache_count << " vendor cache file(s) under " << profiles_path << "\n";
return 0;
}

View File

@@ -28,6 +28,9 @@
#include <cereal/access.hpp>
#include <cereal/types/base_class.hpp>
// The serialize() members below archive ConfigOption hierarchies through
// cereal::base_class, whose registration machinery lives in polymorphic.hpp.
#include <cereal/types/polymorphic.hpp>
namespace Slic3r {
struct FloatOrPercent

View File

@@ -147,6 +147,9 @@ Semver get_version_from_json(std::string file_path)
return Semver();
//throw ConfigurationError(format("Failed loading configuration file \"%1%\": %2%", file_path, err.what()));
}
catch(...) {
return Semver();
}
}
//BBS: add a function to load the key-values from xxx.json
@@ -261,18 +264,28 @@ void extend_default_config_length(DynamicPrintConfig& config, const bool set_nil
}
};
// The four variant sets are immutable after static init and probed for every
// key of every preset loaded; one merged map makes that a single lookup.
// emplace keeps the first insertion, preserving the first-set-wins priority
// of the else-if chain this replaces.
static const std::unordered_map<std::string, int> variant_class = [] {
std::unordered_map<std::string, int> m;
for (const std::string& k : print_options_with_variant) m.emplace(k, 0);
for (const std::string& k : filament_options_with_variant) m.emplace(k, 1);
for (const std::string& k : printer_options_with_variant_1) m.emplace(k, 2);
for (const std::string& k : printer_options_with_variant_2) m.emplace(k, 3);
return m;
}();
for(auto& key :config.keys()){
if(auto iter = print_options_with_variant.find(key); iter != print_options_with_variant.end()){
replace_nil_and_resize(key, process_variant_length);
}
else if(auto iter = filament_options_with_variant.find(key); iter != filament_options_with_variant.end()){
replace_nil_and_resize(key, filament_variant_length);
}
else if(auto iter = printer_options_with_variant_1.find(key); iter != printer_options_with_variant_1.end()){
replace_nil_and_resize(key, machine_variant_length);
}
else if(auto iter = printer_options_with_variant_2.find(key); iter != printer_options_with_variant_2.end()){
replace_nil_and_resize(key, machine_variant_length * 2);
auto iter = variant_class.find(key);
if (iter == variant_class.end())
continue;
switch (iter->second) {
case 0: replace_nil_and_resize(key, process_variant_length); break;
case 1: replace_nil_and_resize(key, filament_variant_length); break;
case 2: replace_nil_and_resize(key, machine_variant_length); break;
case 3: replace_nil_and_resize(key, machine_variant_length * 2); break;
}
}
}

View File

@@ -131,6 +131,10 @@ public:
PrinterVariant() {}
PrinterVariant(const std::string &name) : name(name) {}
std::string name;
// All fields, declaration order — keep in sync; bump CACHE_VERSION on change.
template<class Archive>
void serialize(Archive& ar) { ar(name); } // PrinterVariant
};
struct PrinterModel {
@@ -139,7 +143,7 @@ public:
std::string name;
//BBS: this is internal id for the printer. Currently only used for searching in database
std::string model_id;
PrinterTechnology technology;
PrinterTechnology technology = ptFFF;
std::string family;
std::vector<PrinterVariant> variants;
std::vector<std::string> default_materials;
@@ -162,6 +166,17 @@ public:
}
const PrinterVariant* variant(const std::string &name) const { return const_cast<PrinterModel*>(this)->variant(name); }
// All fields, declaration order — keep in sync; bump CACHE_VERSION on change.
template<class Archive>
void serialize(Archive& ar) // PrinterModel
{
ar(id, name, model_id, technology, family, variants, default_materials,
not_support_bed_types, bed_model, bed_texture, image_bed_type,
bottom_texture_end_name, use_double_extruder_default_texture,
bottom_texture_rect, bottom_texture_rect_longer, middle_texture_rect,
hotend_model);
}
};
std::vector<PrinterModel> models;
@@ -173,6 +188,14 @@ public:
bool valid() const { return ! name.empty() && ! id.empty() && config_version.valid(); }
// All fields, declaration order — keep in sync; bump CACHE_VERSION on change.
template<class Archive>
void serialize(Archive& ar) // VendorProfile
{
ar(name, id, config_version, config_update_url, changelog_url,
models, default_filaments, default_sla_materials);
}
// Load VendorProfile from an ini file.
// If `load_all` is false, only the header with basic info (name, version, URLs) is loaded.
static VendorProfile from_ini(const boost::filesystem::path &path, bool load_all=true);
@@ -427,10 +450,10 @@ public:
Preset(Type type, const std::string &name, bool is_default = false) : type(type), is_default(is_default), name(name) {}
protected:
Preset() = default;
friend class PresetCollection;
friend class PresetBundle;
Preset() = default;
};
bool is_compatible_with_print (const PresetWithVendorProfile &preset, const PresetWithVendorProfile &active_print, const PresetWithVendorProfile &active_printer);

File diff suppressed because it is too large Load Diff

View File

@@ -6,6 +6,7 @@
#include "enum_bitmask.hpp"
#include <memory>
#include <set>
#include <shared_mutex>
#include <unordered_map>
#include <optional>
@@ -170,6 +171,73 @@ struct PresetBundleMetadata
class PresetBundle
{
public:
// ---- Per-vendor preset cache --------------------------------------------
// One cache file per vendor (plus the Orca filament library), stamped with
// the vendor's own profile version rather than a directory scan.
// The cache is not something a caller loads from: a vendor is loaded with
// load_vendor_configs_from_json, which comes from the cache whenever one covers
// it. What is public here is what the cache's own tests drive directly.
// One preset as its JSON subfile states it: the config diff, the name of the
// preset it inherits, and the parse metadata — everything the parse phase of
// load_vendor_configs_from_json extracts and nothing it derives. Inheritance
// is resolved when the entry is installed, against whatever filament library
// is loaded then, so a cache carries no other vendor's values and no other
// vendor's update can make it stale.
// All fields, declaration order — keep in sync; bump CACHE_VERSION on change.
struct CachedPreset
{
std::string name;
std::string sub_path; // path under the vendor's directory
DynamicPrintConfig config_src; // the preset's own diff, nothing inherited
std::string inherits;
std::string description;
std::string instantiation; // "true"/"false" as stated; anything else was already counted as a parse error
std::string setting_id;
std::string filament_id;
std::vector<std::string> renamed_from;
template<class Archive>
void serialize(Archive& ar)
{
ar(name, sub_path, config_src, inherits, description, instantiation,
setting_id, filament_id, renamed_from);
}
};
// Save one vendor (vendor_name at vendor_version): its vendor profile and its
// presets in source form, plus how many errors their parse counted.
static bool save_vendor_cache(const std::string& cache_path, const std::string& vendor_name,
const std::string& vendor_version, const VendorMap& vendors,
const std::vector<CachedPreset>& process_entries,
const std::vector<CachedPreset>& filament_entries,
const std::vector<CachedPreset>& machine_entries,
uint64_t parse_errors);
// Load a validated per-vendor cache into this bundle by installing its
// entries, with base_bundle's filament library as the inheritance base.
// Rejects (returns false, with this bundle left clean) unless the cache
// version, schema fingerprint and vendor name match, the cache was built
// from a vendor profile at least as new as the expected one, and every
// entry installs. An invalid expected version (a profile whose version
// cannot be judged) is never served from cache; Semver::inf() (no profile
// beside the cache at all) accepts whatever is cached.
bool load_vendor_cache(const std::string& cache_path, const std::string& expected_vendor_name,
const Semver& expected_vendor_version, const PresetBundle* base_bundle = nullptr);
// Read the profile version a cache was stamped with, without deserializing its
// presets. Empty if the file is unreadable, not a cache this build understands,
// or not this vendor's. This is how an installed vendor's version is known when
// only its cache is installed.
static std::string peek_vendor_cache_version(const std::string& cache_path, const std::string& expected_vendor_name);
// Enable writing a per-vendor cache after a JSON parse (off by default). Cache
// content is pure parse output, so the guard is policy, not correctness: only
// the deliberate generators (load_system_presets_from_json, the cache build
// tool) write files, not every incidental load a dialog performs.
void set_generate_vendor_caches(bool enable) { m_generate_vendor_caches = enable; }
static DynamicPrintConfig construct_full_config(Preset &in_printer_preset,
Preset &in_print_preset,
const DynamicPrintConfig &project_config,
@@ -444,8 +512,12 @@ public:
/*std::pair<PresetsConfigSubstitutions, size_t> load_configbundle(
const std::string &path, LoadConfigBundleAttributes flags, ForwardCompatibilitySubstitutionRule compatibility_rule);*/
//Orca: load config bundle from json, pass the base bundle to support cross vendor inheritance
// Orca: `dir` is where the vendor is looked for — its own directory, whether or
// not the profile JSONs are still there. A whole-vendor load comes from the
// vendor's preset cache whenever one covers the profile on disk, and is parsed
// from the JSONs (falling back to the ones in resources) only when none does.
std::pair<PresetsConfigSubstitutions, size_t> load_vendor_configs_from_json(
const std::string &path, const std::string &vendor_name, LoadConfigBundleAttributes flags, ForwardCompatibilitySubstitutionRule compatibility_rule, const PresetBundle* base_bundle = nullptr);
const std::string &dir, const std::string &vendor_name, LoadConfigBundleAttributes flags, ForwardCompatibilitySubstitutionRule compatibility_rule, const PresetBundle* base_bundle = nullptr);
// Export a config bundle file containing all the presets and the names of the active presets.
//void export_configbundle(const std::string &path, bool export_system_settings = false, bool export_physical_printers = false);
@@ -517,11 +589,56 @@ public:
// Orca: for validation only.
bool has_errors(bool check_duplicate_filament_subtypes = false) const;
// Errors the last load recorded. What the cache's error accounting promises —
// a cache-served vendor reports what its parse would — is pinned against this.
int error_count() const { return m_errors; }
// Orca: for validation only. Flag any system preset whose inherits / compatible_printers /
// compatible_prints references a deleted (unknown) or renamed (old) preset name.
bool check_preset_references() const;
// Merge one vendor's presets with the other vendor's presets, report duplicates.
// Public so per-vendor-cache consumers (e.g. the setup wizard) can assemble a
// bundle out of several per-vendor caches loaded into separate PresetBundle instances.
std::vector<std::string> merge_presets(PresetBundle &&other);
private:
// Load one vendor from its preset cache: the one in `dir`, or — when that is
// missing or stale — the one shipped in resources/profiles, both judged against
// the vendor as installed in `dir`. False, with this bundle left clean, when
// neither is usable and the vendor has to be parsed. This is how
// load_vendor_configs_from_json reads a cache.
bool load_vendor_cache(const boost::filesystem::path& dir, const std::string& vendor_name, const PresetBundle* base_bundle);
// Read raw cache blob: verify magic, size, CRC.
static bool read_cache_blob(const std::string& path, std::string& out_blob);
// Write a cache blob with the standard 20-byte file header. False when the
// file could not be opened or written whole.
static bool write_cache_blob(const std::string& path, const std::string& blob);
// Install one source-form preset entry into this bundle: resolve `inherits`,
// flatten, validate and register the preset. Returns the reason installation
// failed, empty on success. See the definition for the sharing contract
// between the JSON parse and the cache load.
// retain_configs, when non-null, names the only presets registered into
// config_maps (a full config copy each). The cache load passes the names its
// entries inherit — the only ones ever looked up again; the JSON parse
// retains all, not knowing what later subfiles inherit.
std::string install_vendor_preset(const CachedPreset& entry,
const std::string& path, const std::string& vendor_name,
const PresetBundle* base_bundle,
LoadConfigBundleAttributes flags,
ConfigSubstitutionContext& substitution_context, PresetsConfigSubstitutions& substitutions,
std::map<std::string, DynamicPrintConfig>& config_maps, std::map<std::string, std::string>& filament_id_maps,
PresetCollection* presets_collection, size_t& count, bool is_from_lib,
const std::set<std::string>* retain_configs = nullptr);
// Clear every collection's m_printer_hold_alias, which reset() leaves alone.
void clear_printer_hold_aliases();
// Whether to (re)write a per-vendor cache after a JSON parse.
bool m_generate_vendor_caches { false };
// Orca: validation only - flag any printer with two or more compatible
// filament presets sharing one filament_id (ambiguous AMS subtype match).
bool check_duplicate_filament_subtypes() const;
@@ -529,8 +646,6 @@ private:
//std::pair<PresetsConfigSubstitutions, std::string> load_system_presets(ForwardCompatibilitySubstitutionRule compatibility_rule);
//BBS: add json related logic
std::pair<PresetsConfigSubstitutions, std::string> load_system_presets_from_json(ForwardCompatibilitySubstitutionRule compatibility_rule);
// Merge one vendor's presets with the other vendor's presets, report duplicates.
std::vector<std::string> merge_presets(PresetBundle &&other);
// Update the multicolor information for filaments.
void update_filament_multi_color();
// Update renamed_from and alias maps of system profiles.
@@ -565,6 +680,38 @@ private:
ENABLE_ENUM_BITMASK_OPERATORS(PresetBundle::LoadConfigBundleAttribute)
// True if `vendor` is installed in data_dir()/system. A build that ships preset
// caches installs the cache alone, so it — not the profile — marks a vendor
// installed, and either one on its own counts.
extern bool is_vendor_installed(const std::string& vendor);
// The version of the installed vendor: what its profile claims, or what its cache
// was stamped with where only the cache is installed. Invalid Semver if neither is.
extern Semver installed_vendor_version(const std::string& vendor);
// Remove every form `vendor` can be installed as from data_dir()/system: its
// profile, its preset cache, and its preset directory.
extern void remove_installed_vendor(const std::string& vendor);
// The vendors `dir` holds, sorted: one is named by its profile or, in a build that
// ships preset caches instead of the raw profile JSONs, by its cache alone.
extern std::set<std::string> vendor_names_in(const boost::filesystem::path& dir);
// The version a build ships `vendor` at: whichever of its preset cache and its
// profile is newer, that being the one installing lays down. Invalid Semver if the
// build ships neither.
extern Semver resource_vendor_version(const std::string& vendor);
// Install vendors from the resources directory into the data directory, each as
// its preset cache or as its profile and preset JSONs — whichever of the two the
// build ships at the newer version. Anything the previous install of that vendor
// left behind goes, so only the form just installed is there to be loaded.
// bundle_names: vendor names, without extension.
// Returns false on the first vendor that cannot be installed.
extern bool install_vendor_bundles_from_resources(const std::vector<std::string>& bundle_names,
const std::string& resource_subdir = "profiles",
const std::string& data_subdir = "system");
} // namespace Slic3r
#endif /* slic3r_PresetBundle_hpp_ */

View File

@@ -2412,7 +2412,8 @@ namespace cereal {
archive(serialization_key_ordinal);
assert(serialization_key_ordinal > 0);
auto it = Slic3r::print_config_def.by_serialization_key_ordinal.find(serialization_key_ordinal);
assert(it != Slic3r::print_config_def.by_serialization_key_ordinal.end());
if (it == Slic3r::print_config_def.by_serialization_key_ordinal.end())
throw std::runtime_error("VendorCache: unknown serialization_key_ordinal " + std::to_string(serialization_key_ordinal) + " - cache is stale");
config.set_key_value(it->second->opt_key, it->second->load_option_from_archive(archive));
}
}

View File

@@ -190,6 +190,19 @@ public:
os << self.to_string();
return os;
}
// cereal: round-trip through the standard 3-part string (major.minor.patch).
// to_string() uses a BBS 4-part format that semver_parse() cannot read back.
template<class Archive>
std::string save_minimal(const Archive&) const { return to_string_sf(); }
template<class Archive>
void load_minimal(const Archive&, const std::string& s) {
auto v = Semver::parse(s);
if (! v)
throw std::runtime_error("Semver: cannot parse serialized version: " + s);
*this = std::move(*v);
}
private:
semver_t ver;

View File

@@ -722,15 +722,6 @@ void copy_directory_recursively(const boost::filesystem::path& source,
std::function<bool(const std::string)> filter = nullptr,
bool merge_mode = false);
// Install vendor bundles from resources directory to data directory
// bundle_names: vector of vendor bundle names (without .json extension)
// resource_subdir: subdirectory under resources_dir() (default: "profiles")
// data_subdir: subdirectory under data_dir() (default: "system")
// Returns: true if all bundles installed successfully, false otherwise
bool install_vendor_bundles_from_resources(const std::vector<std::string>& bundle_names,
const std::string& resource_subdir = "profiles",
const std::string& data_subdir = "system");
// Orca: Since 1.7.9 Boost deprecated save_string_file and load_string_file, copy and modified from boost 1.7.8
void save_string_file(const boost::filesystem::path& p, const std::string& str);
void load_string_file(const boost::filesystem::path& p, std::string& str);

View File

@@ -1724,76 +1724,6 @@ void copy_directory_recursively(const boost::filesystem::path& source,
return;
}
bool install_vendor_bundles_from_resources(
const std::vector<std::string>& bundle_names,
const std::string& resource_subdir,
const std::string& data_subdir)
{
namespace fs = boost::filesystem;
fs::path rsrc_path = fs::path(Slic3r::resources_dir()) / resource_subdir;
fs::path vendor_path = fs::path(Slic3r::data_dir()) / data_subdir;
BOOST_LOG_TRIVIAL(info) << "Installing " << bundle_names.size() << " bundles from resources...";
for (const auto &bundle : bundle_names) {
try {
// Install the JSON file
auto path_in_rsrc = (rsrc_path / bundle).replace_extension(".json");
auto path_in_vendors = (vendor_path / bundle).replace_extension(".json");
if (!fs::exists(path_in_rsrc)) {
BOOST_LOG_TRIVIAL(warning) << "Bundle not found in resources: " << bundle;
return false;
}
// Create target directory if needed
if (!fs::exists(vendor_path))
fs::create_directories(vendor_path);
// Copy JSON file
std::string error_message;
CopyFileResult cfr = copy_file(path_in_rsrc.string(), path_in_vendors.string(), error_message, false);
if (cfr != CopyFileResult::SUCCESS) {
BOOST_LOG_TRIVIAL(error) << "Failed to copy " << bundle << ".json: " << error_message;
return false;
}
// Copy the vendor directory (if it exists)
auto dir_in_rsrc = rsrc_path / bundle;
auto dir_in_vendors = vendor_path / bundle;
if (fs::exists(dir_in_rsrc) && fs::is_directory(dir_in_rsrc)) {
// Remove existing directory
if (fs::exists(dir_in_vendors))
fs::remove_all(dir_in_vendors);
fs::create_directories(dir_in_vendors);
// Copy with file filter (same as PresetUpdater::install_bundles_rsrc)
// Filter out certain file types: .stl, .png, .svg, .jpeg, .jpg, .3mf
auto file_filter = [](const std::string name) -> bool {
return boost::iends_with(name, ".stl") ||
boost::iends_with(name, ".png") ||
boost::iends_with(name, ".svg") ||
boost::iends_with(name, ".jpeg") ||
boost::iends_with(name, ".jpg") ||
boost::iends_with(name, ".3mf");
};
copy_directory_recursively(dir_in_rsrc, dir_in_vendors, file_filter);
}
BOOST_LOG_TRIVIAL(info) << "Successfully installed bundle: " << bundle;
} catch (const std::exception& e) {
BOOST_LOG_TRIVIAL(error) << "Exception installing bundle " << bundle << ": " << e.what();
return false;
}
}
return true;
}
void save_string_file(const boost::filesystem::path& p, const std::string& str)
{
boost::nowide::ofstream file;

View File

@@ -432,14 +432,9 @@ const Snapshot& SnapshotDB::take_snapshot(const AppConfig &app_config, Snapshot:
cfg.models_variants_installed.erase(it ++);
else
++ it;
// Read the active config bundle, parse the config version.
PresetBundle bundle;
//BBS: change directoties by design
//bundle.load_configbundle((data_dir / PRESET_SYSTEM_DIR / (cfg.name + ".ini")).string(), PresetBundle::LoadConfigBundleAttribute::LoadVendorOnly, ForwardCompatibilitySubstitutionRule::EnableSilent);
bundle.load_vendor_configs_from_json((data_dir/PRESET_SYSTEM_DIR).string(), cfg.name, PresetBundle::LoadConfigBundleAttribute::LoadVendorOnly, ForwardCompatibilitySubstitutionRule::EnableSilent);
for (const auto &vp : bundle.vendors)
if (vp.second.id == cfg.name)
cfg.version.config_version = vp.second.config_version;
// Orca: the version the vendor is installed at, read from its profile or —
// where the cache is the whole installation — from the cache's own stamp.
cfg.version.config_version = installed_vendor_version(cfg.name);
snapshot.vendor_configs.emplace_back(std::move(cfg));
}

View File

@@ -66,41 +66,41 @@ using Config::SnapshotDB;
// Configuration data structures extensions needed for the wizard
//BBS: set BBL as default
bool Bundle::load(fs::path source_path, bool ais_in_resources, bool ais_bbl_bundle)
bool Bundle::load(fs::path dir, const std::string &vendor_name, bool ais_in_resources, bool ais_bbl_bundle)
{
this->preset_bundle = std::make_unique<PresetBundle>();
this->is_in_resources = ais_in_resources;
this->is_bbl_bundle = ais_bbl_bundle;
std::string path_string = source_path.string();
std::string parent_path = source_path.parent_path().string();
//BBS: add json logic for vendor bundles
std::string vendor_name = source_path.filename().string();
if (Slic3r::is_json_file(path_string)) {
// Remove the .json suffix.
vendor_name.erase(vendor_name.size() - 5);
}
else
// Orca: served from the vendor's preset cache where one covers it — which is
// how a shipped build carries its vendors — and parsed from the JSONs otherwise.
// A vendor that can be neither read nor parsed — a cache the build cannot use
// with the preset JSONs behind it pruned, say — is one the wizard cannot offer.
// Every other vendor still can be, so it is left out rather than thrown over.
size_t presets_loaded = 0;
try {
auto [config_substitutions, loaded] = preset_bundle->load_vendor_configs_from_json(
dir.string(), vendor_name, PresetBundle::LoadConfigBundleAttribute::LoadSystem, ForwardCompatibilitySubstitutionRule::Disable);
UNUSED(config_substitutions);
// No substitutions shall be reported when loading a system config bundle, no substitutions are allowed.
assert(config_substitutions.empty());
presets_loaded = loaded;
} catch (const std::exception &e) {
BOOST_LOG_TRIVIAL(fatal) << boost::format("Vendor bundle: `%1%`: cannot be loaded, leaving it out: %2%") % vendor_name % e.what();
return false;
// Throw when parsing invalid configuration. Only valid configuration is supposed to be provided over the air.
//BBS: add json logic for vendor bundles
auto [config_substitutions, presets_loaded] = preset_bundle->load_vendor_configs_from_json(
parent_path, vendor_name, PresetBundle::LoadConfigBundleAttribute::LoadSystem, ForwardCompatibilitySubstitutionRule::Disable);
UNUSED(config_substitutions);
// No substitutions shall be reported when loading a system config bundle, no substitutions are allowed.
assert(config_substitutions.empty());
}
auto first_vendor = preset_bundle->vendors.begin();
if (first_vendor == preset_bundle->vendors.end()) {
BOOST_LOG_TRIVIAL(error) << boost::format("Vendor bundle: `%1%`: No vendor information defined, cannot install.") % path_string;
BOOST_LOG_TRIVIAL(error) << boost::format("Vendor bundle: `%1%`: No vendor information defined, cannot install.") % vendor_name;
return false;
}
if (presets_loaded == 0) {
BOOST_LOG_TRIVIAL(error) << boost::format("Vendor bundle: `%1%`: No profile loaded.") % path_string;
BOOST_LOG_TRIVIAL(error) << boost::format("Vendor bundle: `%1%`: No profile loaded.") % vendor_name;
return false;
}
}
BOOST_LOG_TRIVIAL(trace) << boost::format("Vendor bundle: `%1%`: %2% profiles loaded.") % path_string % presets_loaded;
BOOST_LOG_TRIVIAL(trace) << boost::format("Vendor bundle: `%1%`: %2% profiles loaded.") % vendor_name % presets_loaded;
this->vendor_profile = &first_vendor->second;
return true;
}
@@ -125,15 +125,10 @@ BundleMap BundleMap::load()
//Orca: add custom as default
//Orca: add json logic for vendor bundle
auto orca_bundle_path = (vendor_dir / PresetBundle::ORCA_DEFAULT_BUNDLE).replace_extension(".json");
auto orca_bundle_rsrc = false;
if (!boost::filesystem::exists(orca_bundle_path)) {
orca_bundle_path = (rsrc_vendor_dir / PresetBundle::ORCA_DEFAULT_BUNDLE).replace_extension(".json");
orca_bundle_rsrc = true;
}
{
const bool from_rsrc = ! is_vendor_installed(PresetBundle::ORCA_DEFAULT_BUNDLE);
Bundle bbl_bundle;
if (bbl_bundle.load(std::move(orca_bundle_path), orca_bundle_rsrc, true))
if (bbl_bundle.load(from_rsrc ? rsrc_vendor_dir : vendor_dir, PresetBundle::ORCA_DEFAULT_BUNDLE, from_rsrc, true))
res.emplace(PresetBundle::ORCA_DEFAULT_BUNDLE, std::move(bbl_bundle));
}
@@ -141,18 +136,13 @@ BundleMap BundleMap::load()
// and then additionally from resources/profiles.
bool is_in_resources = false;
for (auto dir : { &vendor_dir, &rsrc_vendor_dir }) {
for (const auto &dir_entry : boost::filesystem::directory_iterator(*dir)) {
//BBS: add json logic for vendor bundle
if (Slic3r::is_json_file(dir_entry.path().string())) {
std::string id = dir_entry.path().stem().string(); // stem() = filename() without the trailing ".json" part
for (const std::string &id : vendor_names_in(*dir)) {
// Don't load this bundle if we've already loaded it.
if (res.find(id) != res.end()) { continue; }
// Don't load this bundle if we've already loaded it.
if (res.find(id) != res.end()) { continue; }
Bundle bundle;
if (bundle.load(dir_entry.path(), is_in_resources))
res.emplace(std::move(id), std::move(bundle));
}
Bundle bundle;
if (bundle.load(*dir, id, is_in_resources))
res.emplace(id, std::move(bundle));
}
is_in_resources = true;

View File

@@ -71,9 +71,11 @@ struct Bundle
Bundle() = default;
Bundle(Bundle&& other);
// Load the vendor `vendor_name` as it is installed in `dir`, from its preset
// cache or its profile JSONs, whichever is usable.
// Returns false if not loaded. Reason for that is logged as boost::log error.
//BBS: set BBL as default
bool load(fs::path source_path, bool is_in_resources, bool is_bbl_bundle = false);
bool load(fs::path dir, const std::string &vendor_name, bool is_in_resources, bool is_bbl_bundle = false);
const std::string& vendor_id() const { return vendor_profile->id; }
};

View File

@@ -2201,25 +2201,14 @@ bool CreatePrinterPresetDialog::load_system_and_user_presets_with_curr_model(Pre
} else {
selected_vendor_id = m_printer_preset_vendor_selected.id;
if (boost::filesystem::exists(boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR / selected_vendor_id)) {
preset_path = (boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR).string();
} else if (boost::filesystem::exists(boost::filesystem::path(Slic3r::resources_dir()) / "profiles" / selected_vendor_id)) {
preset_path = (boost::filesystem::path(Slic3r::resources_dir()) / "profiles").string();
}
if (preset_path.empty()) {
BOOST_LOG_TRIVIAL(info) << "Preset path was not found";
MessageDialog dlg(this, _L("Preset path was not found; please reselect vendor."), wxString(SLIC3R_APP_FULL_NAME) + " - " + _L("Info"),
wxYES_NO | wxYES_DEFAULT | wxCENTRE);
dlg.ShowModal();
return false;
}
try {
// Pass the app's preset bundle (which already holds OrcaFilamentLibrary) as the base
// bundle so vendor filaments that inherit OFL bases resolve via the existing
// cross-vendor inheritance path.
temp_preset_bundle.load_vendor_configs_from_json(preset_path, selected_vendor_id,
// Orca: served from the vendor's preset cache where one covers it — a shipped
// build carries that instead of the raw preset JSONs — and parsed otherwise.
temp_preset_bundle.load_vendor_configs_from_json((boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR).string(),
selected_vendor_id,
PresetBundle::LoadConfigBundleAttribute::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent,
wxGetApp().preset_bundle);

View File

@@ -1,7 +1,9 @@
#include "WebGuideDialog.hpp"
#include "ConfigWizard.hpp"
#include <boost/algorithm/string/join.hpp>
#include <boost/filesystem/operations.hpp>
#include <boost/nowide/fstream.hpp>
#include <boost/filesystem/path.hpp>
#include <boost/iostreams/detail/select.hpp>
#include <boost/log/trivial.hpp>
@@ -9,6 +11,7 @@
#include "I18N.hpp"
#include "libslic3r/AppConfig.hpp"
#include "libslic3r/Config.hpp"
#include "libslic3r/Preset.hpp"
#include "libslic3r/PresetBundle.hpp"
#include "slic3r/GUI/wxExtensions.hpp"
#include "slic3r/GUI/GUI_App.hpp"
@@ -41,8 +44,6 @@ using namespace nlohmann;
namespace Slic3r { namespace GUI {
json m_ProfileJson;
static wxString update_custom_filaments()
{
json m_Res = json::object();
@@ -190,12 +191,10 @@ GuideFrame::GuideFrame(GUI_App *pGUI, long style)
GuideFrame::~GuideFrame()
{
m_destroy = true;
if (m_load_task && m_load_task->joinable()) {
*m_cancel_token = true; // stop the loading thread and any queued CallAfter lambdas before join
if (m_load_task && m_load_task->joinable())
m_load_task->join();
delete m_load_task;
m_load_task = nullptr;
}
m_load_task.reset();
if (m_browser) {
delete m_browser;
m_browser = nullptr;
@@ -301,15 +300,71 @@ void GuideFrame::OnNavigationRequest(wxWebViewEvent &evt)
/**
* Callback invoked when a navigation request was accepted
*/
// The empty shape every profile-loading path starts from or falls back to.
void GuideFrame::reset_profile_json()
{
m_ProfileJson["model"] = json::array();
m_ProfileJson["machine"] = json::object();
m_ProfileJson["filament"] = json::object();
m_ProfileJson["process"] = json::array();
}
void GuideFrame::init_guide_paths()
{
m_ProfileJson = json::parse("{}");
reset_profile_json();
vendor_dir = (boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR).make_preferred();
rsrc_vendor_dir = (boost::filesystem::path(resources_dir()) / "profiles").make_preferred();
orca_bundle_rsrc = true;
if (boost::filesystem::exists(vendor_dir)) {
for (const auto& entry : boost::filesystem::directory_iterator(vendor_dir)) {
if (!boost::filesystem::is_directory(entry) &&
boost::iequals(entry.path().extension().string(), ".json") &&
!boost::iequals(entry.path().stem().string(), PresetBundle::ORCA_FILAMENT_LIBRARY)) {
orca_bundle_rsrc = false;
break;
}
}
}
auto lib_json = boost::filesystem::path(PresetBundle::ORCA_FILAMENT_LIBRARY).replace_extension(".json");
m_OrcaFilaLibPath = boost::filesystem::exists(vendor_dir / lib_json)
? (vendor_dir / PresetBundle::ORCA_FILAMENT_LIBRARY).string()
: (rsrc_vendor_dir / PresetBundle::ORCA_FILAMENT_LIBRARY).string();
}
void GuideFrame::on_profile_loaded()
{
// Must be called on the main thread.
SaveProfileData();
const std::string strAll = m_ProfileJson.dump(-1, ' ', false, json::error_handler_t::ignore);
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << ", finished, json contents:\n" << strAll;
json res;
res["command"] = "userguide_profile_load_finish";
res["sequence_id"] = "10001";
RunScript(wxString::Format("HandleStudio(%s)", res.dump(-1, ' ', true)));
}
void GuideFrame::OnNavigationComplete(wxWebViewEvent &evt)
{
//wxLogMessage("%s", "Navigation complete; url='" + evt.GetURL() + "'");
if (!bFirstComplete) {
m_load_task = new boost::thread(boost::bind(&GuideFrame::LoadProfileData, this));
// boost::thread LoadProfileThread(boost::bind(&GuideFrame::LoadProfileData, this));
//LoadProfileThread.detach();
bFirstComplete = true;
try {
init_guide_paths();
if (BuildProfileDataFromPresetBundle()) {
if (!*m_cancel_token)
on_profile_loaded();
} else {
// Presets not yet in memory — delegate to background thread.
m_load_task = std::make_unique<boost::thread>(boost::bind(&GuideFrame::LoadProfileData, this));
}
} catch (const std::exception& e) {
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << ", init error: " << e.what();
m_load_task = std::make_unique<boost::thread>(boost::bind(&GuideFrame::LoadProfileData, this));
}
}
m_browser->Show();
@@ -762,11 +817,9 @@ bool GuideFrame::apply_config(AppConfig *app_config, PresetBundle *preset_bundle
bool check_unsaved_preset_changes = false;
std::vector<std::string> install_bundles;
std::vector<std::string> remove_bundles;
const auto vendor_dir = (boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR).make_preferred();
for (const auto &it : enabled_vendors) {
if (it.second.size() > 0) {
auto vendor_file = vendor_dir/(it.first + ".json");
if (!fs::exists(vendor_file)) {
if (!is_vendor_installed(it.first)) {
install_bundles.emplace_back(it.first);
}
}
@@ -777,8 +830,7 @@ bool GuideFrame::apply_config(AppConfig *app_config, PresetBundle *preset_bundle
if (it.second.size() > 0) {
if (enabled_vendors.find(it.first) != enabled_vendors.end())
continue;
auto vendor_file = vendor_dir/(it.first + ".json");
if (fs::exists(vendor_file)) {
if (is_vendor_installed(it.first)) {
remove_bundles.emplace_back(it.first);
}
}
@@ -1127,99 +1179,244 @@ int GuideFrame::GetFilamentInfo( std::string VendorDirectory, json & pFilaList,
return status;
}
int GuideFrame::LoadProfileData()
bool GuideFrame::BuildProfileJson(const PresetBundle& bundle, bool require_all_resource_vendors)
{
try {
m_ProfileJson = json::parse("{}");
m_ProfileJson["model"] = json::array();
m_ProfileJson["machine"] = json::object();
m_ProfileJson["filament"] = json::object();
m_ProfileJson["process"] = json::array();
// Models from vendor profiles
for (const auto& [vendor_id, vp] : bundle.vendors) {
for (const auto& model : vp.models) {
std::string nozzle_str;
for (const auto& v : model.variants) {
if (!nozzle_str.empty()) nozzle_str += ";";
nozzle_str += v.name;
}
const std::string materials_str = boost::algorithm::join(model.default_materials, ";");
boost::filesystem::path cover_path =
(boost::filesystem::path(resources_dir()) / "profiles" / vp.id / (model.id + "_cover.png"))
.make_preferred();
if (!boost::filesystem::exists(cover_path))
cover_path =
(boost::filesystem::path(resources_dir()) / "web/image/printer" / (model.id + "_cover.png"))
.make_preferred();
vendor_dir = (boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR).make_preferred();
rsrc_vendor_dir = (boost::filesystem::path(resources_dir()) / "profiles").make_preferred();
// Orca: add custom as default
// Orca: add json logic for vendor bundle
orca_bundle_rsrc = true;
// search if there exists a .json file in vendor_dir folder, if exists, set orca_bundle_rsrc to false
for (const auto& entry : boost::filesystem::directory_iterator(vendor_dir)) {
if (!boost::filesystem::is_directory(entry) && boost::iequals(entry.path().extension().string(), ".json") && !boost::iequals(entry.path().stem().string(), PresetBundle::ORCA_FILAMENT_LIBRARY)) {
orca_bundle_rsrc = false;
break;
json entry;
entry["model"] = model.id;
entry["name"] = model.name;
entry["vendor"] = vp.id;
entry["nozzle_diameter"] = nozzle_str;
entry["materials"] = materials_str;
entry["cover"] = cover_path.string();
entry["nozzle_selected"] = "";
entry["sub_path"] = "";
m_ProfileJson["model"].push_back(entry);
}
}
// load the default filament library first
std::set<std::string> loaded_vendors;
auto filament_library_name = boost::filesystem::path(PresetBundle::ORCA_FILAMENT_LIBRARY).replace_extension(".json");
if (boost::filesystem::exists(vendor_dir / filament_library_name)) {
m_OrcaFilaLibPath = (vendor_dir / PresetBundle::ORCA_FILAMENT_LIBRARY).string();
LoadProfileFamily(PresetBundle::ORCA_FILAMENT_LIBRARY, (vendor_dir / filament_library_name).string());
} else {
m_OrcaFilaLibPath = (rsrc_vendor_dir / PresetBundle::ORCA_FILAMENT_LIBRARY).string();
LoadProfileFamily(PresetBundle::ORCA_FILAMENT_LIBRARY, (rsrc_vendor_dir / filament_library_name).string());
}
loaded_vendors.insert(PresetBundle::ORCA_FILAMENT_LIBRARY);
// Machine map: preset name -> {model, nozzle variant}
for (const Preset& p : bundle.printers()) {
if (!p.is_system || !p.vendor) continue;
const auto* printer_model = p.config.option<ConfigOptionString>("printer_model");
const auto* printer_variant = p.config.option<ConfigOptionString>("printer_variant");
if (!printer_model || printer_model->value.empty() || !printer_variant) continue;
//load custom bundle from user data path
boost::filesystem::directory_iterator endIter;
for (boost::filesystem::directory_iterator iter(vendor_dir); iter != endIter; iter++) {
if (!boost::filesystem::is_directory(*iter)) {
wxString strVendor = from_u8(iter->path().string()).BeforeLast('.');
strVendor = strVendor.AfterLast('\\');
strVendor = strVendor.AfterLast('/');
wxString strExtension = from_u8(iter->path().string()).AfterLast('.').Lower();
if(strExtension.CmpNoCase("json") != 0 || loaded_vendors.find(w2s(strVendor)) != loaded_vendors.end())
continue;
LoadProfileFamily(w2s(strVendor), iter->path().string());
loaded_vendors.insert(w2s(strVendor));
}
if (m_destroy)
return 0;
json mach;
mach["model"] = printer_model->value;
mach["nozzle"] = printer_variant->value;
m_ProfileJson["machine"][p.name] = mach;
}
boost::filesystem::directory_iterator others_endIter;
for (boost::filesystem::directory_iterator iter(rsrc_vendor_dir); iter != others_endIter; iter++) {
if (!boost::filesystem::is_directory(*iter)) {
wxString strVendor = from_u8(iter->path().string()).BeforeLast('.');
strVendor = strVendor.AfterLast('\\');
strVendor = strVendor.AfterLast('/');
wxString strExtension = from_u8(iter->path().string()).AfterLast('.').Lower();
if (strExtension.CmpNoCase("json") != 0 || loaded_vendors.find(w2s(strVendor)) != loaded_vendors.end())
continue;
// Filament map from system filament presets (vendor/type already resolved in config)
const json& machines = m_ProfileJson["machine"];
for (const Preset& p : bundle.filaments()) {
if (!p.is_system || !p.vendor) continue;
const auto* fila_vendor = p.config.option<ConfigOptionStrings>("filament_vendor");
const auto* fila_type = p.config.option<ConfigOptionStrings>("filament_type");
const auto* compat_printers = p.config.option<ConfigOptionStrings>("compatible_printers");
LoadProfileFamily(w2s(strVendor), iter->path().string());
loaded_vendors.insert(w2s(strVendor));
std::string vendor = (fila_vendor && !fila_vendor->values.empty()) ? fila_vendor->values[0] : "";
std::string type = (fila_type && !fila_type->values.empty()) ? fila_type->values[0] : "";
std::string model_list;
if (compat_printers) {
for (const std::string& pname : compat_printers->values) {
auto it = machines.find(pname);
if (it != machines.end()) {
const std::string m = (*it)["model"];
const std::string n = (*it)["nozzle"];
model_list += "[" + m + "++" + n + "]";
}
}
}
if (m_destroy)
return 0;
json ff;
ff["name"] = p.name;
ff["sub_path"] = p.file;
ff["vendor"] = vendor;
ff["type"] = type;
ff["models"] = model_list;
ff["selected"] = 0;
m_ProfileJson["filament"][p.name] = ff;
}
wxGetApp().CallAfter([this] {
if (!m_destroy) {
//sync to appconfig first to populate current selections
SaveProfileData();
// Process list from visible system print presets
for (const Preset& p : bundle.prints()) {
if (!p.is_system || !p.vendor || !p.is_visible) continue;
json entry;
entry["name"] = p.name;
entry["sub_path"] = p.file;
m_ProfileJson["process"].push_back(entry);
}
//sync to web after selections are populated
std::string strAll = m_ProfileJson.dump(-1, ' ', false, json::error_handler_t::ignore);
if (require_all_resource_vendors) {
// If rsrc_vendor_dir has vendors (profile JSONs, or the preset caches a
// packaged build ships instead) not covered by the current bundle, the
// bundle is incomplete (e.g. dev env where data_dir/system only has
// OrcaFilamentLibrary+Custom). Fall back so the slow path reads both dirs.
try {
for (const std::string& name : vendor_names_in(rsrc_vendor_dir)) {
if (bundle.vendors.find(name) == bundle.vendors.end()) {
BOOST_LOG_TRIVIAL(info) << "GuideFrame: vendor '" << name
<< "' in resources but not in preset_bundle — falling back to JSON loading";
reset_profile_json();
return false;
}
}
} catch (const std::exception&) {}
}
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << ", finished, json contents: " << std::endl << strAll;
json m_Res = json::object();
m_Res["command"] = "userguide_profile_load_finish";
m_Res["sequence_id"] = "10001";
wxString strJS = wxString::Format("HandleStudio(%s)", m_Res.dump(-1, ' ', true));
BOOST_LOG_TRIVIAL(info) << "GuideFrame: built profile data ("
<< m_ProfileJson["model"].size() << " models, "
<< m_ProfileJson["machine"].size() << " machines, "
<< m_ProfileJson["filament"].size() << " filaments)";
return !m_ProfileJson["machine"].empty();
} catch (const std::exception& e) {
BOOST_LOG_TRIVIAL(warning) << "GuideFrame::BuildProfileJson failed: " << e.what()
<< " — falling back to JSON loading";
reset_profile_json();
return false;
}
}
RunScript(strJS);
bool GuideFrame::BuildProfileDataFromPresetBundle()
{
PresetBundle* pb = wxGetApp().preset_bundle;
if (!pb || pb->vendors.empty())
return false;
return BuildProfileJson(*pb, /*require_all_resource_vendors=*/true);
}
bool GuideFrame::BuildProfileDataFromVendors()
{
try {
// Same vendor set and precedence as the JSON scan in LoadProfileData: a
// vendor in the user's system dir shadows the bundled one of that name.
// A vendor is named by its profile or, where a build ships preset caches
// instead, by its cache alone — so both forms name one here.
std::map<std::string, boost::filesystem::path> vendor_files;
auto collect = [&vendor_files](const boost::filesystem::path& dir) {
boost::system::error_code ec;
if (!boost::filesystem::exists(dir, ec))
return;
for (const auto& e : boost::filesystem::directory_iterator(dir, ec))
if (Slic3r::is_json_file(e.path().string()) || e.path().extension() == ".opc")
vendor_files.emplace(e.path().stem().string(), e.path()); // first wins
};
collect(vendor_dir);
collect(rsrc_vendor_dir);
// Each vendor comes from its preset cache where one covers it, which is what
// makes this worth doing instead of the scan below; the filament library goes
// first because the others' filaments inherit from it, and resolving those on
// the vendors the cache does not cover needs it already loaded.
PresetBundle bundle;
auto load_vendor = [this](PresetBundle& into, const std::string& vendor, const PresetBundle* base) {
into.load_vendor_configs_from_json(vendor_dir.string(), vendor, PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent, base);
};
const std::string filament_library(PresetBundle::ORCA_FILAMENT_LIBRARY);
if (vendor_files.count(filament_library))
load_vendor(bundle, filament_library, nullptr);
for (const auto& entry : vendor_files) {
if (*m_cancel_token)
return false; // as in the scan below: a vendor without a cache is parsed, and that takes time
const std::string& vendor = entry.first;
// A cache is only ever written for a versioned vendor; a JSON has to be
// asked, so that an unversioned one (blacklist.json) carrying no presets
// is passed over.
if (vendor == filament_library ||
(entry.second.extension() != ".opc" && ! get_version_from_json(entry.second.string()).valid()))
continue;
PresetBundle tmp;
load_vendor(tmp, vendor, &bundle);
bundle.merge_presets(std::move(tmp));
}
if (bundle.vendors.empty())
return false;
return BuildProfileJson(bundle, /*require_all_resource_vendors=*/false);
} catch (const std::exception& e) {
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << " failed: " << e.what();
reset_profile_json();
return false;
}
}
int GuideFrame::LoadProfileData()
{
// Background thread: the fast path in OnNavigationComplete failed (presets not yet loaded).
// Loading order (fastest to slowest):
// 1. Load every vendor, from its preset cache wherever one covers it
// 2. Read all vendor JSONs by hand
try {
if (!BuildProfileDataFromVendors()) {
// Last resort — read all vendor JSONs
std::set<std::string> loaded_vendors;
auto filament_library_name = boost::filesystem::path(PresetBundle::ORCA_FILAMENT_LIBRARY).replace_extension(".json");
if (boost::filesystem::exists(vendor_dir / filament_library_name))
LoadProfileFamily(PresetBundle::ORCA_FILAMENT_LIBRARY, (vendor_dir / filament_library_name).string());
else
LoadProfileFamily(PresetBundle::ORCA_FILAMENT_LIBRARY, (rsrc_vendor_dir / filament_library_name).string());
loaded_vendors.insert(PresetBundle::ORCA_FILAMENT_LIBRARY);
boost::filesystem::directory_iterator endIter;
for (boost::filesystem::directory_iterator iter(vendor_dir); iter != endIter; iter++) {
if (!boost::filesystem::is_directory(*iter)) {
wxString strVendor = from_u8(iter->path().string()).BeforeLast('.');
strVendor = strVendor.AfterLast('\\');
strVendor = strVendor.AfterLast('/');
wxString strExtension = from_u8(iter->path().string()).AfterLast('.').Lower();
if (strExtension.CmpNoCase("json") != 0 || loaded_vendors.find(w2s(strVendor)) != loaded_vendors.end())
continue;
LoadProfileFamily(w2s(strVendor), iter->path().string());
loaded_vendors.insert(w2s(strVendor));
}
if (*m_cancel_token) return 0;
}
boost::filesystem::directory_iterator others_endIter;
for (boost::filesystem::directory_iterator iter(rsrc_vendor_dir); iter != others_endIter; iter++) {
if (!boost::filesystem::is_directory(*iter)) {
wxString strVendor = from_u8(iter->path().string()).BeforeLast('.');
strVendor = strVendor.AfterLast('\\');
strVendor = strVendor.AfterLast('/');
wxString strExtension = from_u8(iter->path().string()).AfterLast('.').Lower();
if (strExtension.CmpNoCase("json") != 0 || loaded_vendors.find(w2s(strVendor)) != loaded_vendors.end())
continue;
LoadProfileFamily(w2s(strVendor), iter->path().string());
loaded_vendors.insert(w2s(strVendor));
}
if (*m_cancel_token) return 0;
}
}
// Capture the cancel token by value (shared_ptr) so the lambda doesn't
// touch `this` if GuideFrame is destroyed before the event fires.
auto tok = m_cancel_token;
wxGetApp().CallAfter([this, tok] {
if (!*tok)
on_profile_loaded();
});
} catch (std::exception& e) {
// wxLogMessage("GUIDE: load_profile_error %s ", e.what());
// wxMessageBox(e.what(), "", MB_OK);
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << ", error: " << e.what() << std::endl;
} catch (const std::exception& e) {
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << ", error: " << e.what();
}
filament_info_cache.clear();

View File

@@ -30,10 +30,14 @@
#include "libslic3r/PresetBundle.hpp"
#include "slic3r/Utils/PresetUpdater.hpp"
#include <atomic>
#include <memory>
#include <unordered_map>
#include <nlohmann/json.hpp>
#include <boost/thread.hpp>
namespace Slic3r { namespace GUI {
class GuideFrame : public DPIDialog
@@ -78,6 +82,12 @@ public:
int LoadProfileData();
int SaveProfileData();
int LoadProfileFamily(std::string strVendor, std::string strFilePath);
void init_guide_paths();
void on_profile_loaded();
bool BuildProfileJson(const PresetBundle& bundle, bool require_all_resource_vendors);
bool BuildProfileDataFromPresetBundle();
bool BuildProfileDataFromVendors();
void reset_profile_json();
int SaveProfile();
int GetFilamentInfo( std::string VendorDirectory,json & pFilaList, std::string filepath, std::string &sVendor, std::string &sType);
@@ -112,8 +122,11 @@ private:
//First Load
bool bFirstComplete{false};
bool m_destroy{false};
boost::thread* m_load_task{ nullptr };
// Set once in the destructor. Read through `this` by the loading thread
// (joined before `this` dies) and captured as the shared_ptr by CallAfter
// lambdas so they don't touch `this` after the object is freed.
std::shared_ptr<std::atomic<bool>> m_cancel_token{std::make_shared<std::atomic<bool>>(false)};
std::unique_ptr<boost::thread> m_load_task;
// User Config
bool PrivacyUse;
@@ -123,6 +136,7 @@ private:
bool InstallNetplugin;
bool network_plugin_ready {false};
json m_ProfileJson;
json m_OrcaFilaList;
std::string m_OrcaFilaLibPath;

View File

@@ -1044,46 +1044,42 @@ void PresetUpdater::priv::check_installed_vendor_profiles() const
std::set<std::string> bundles;
// Orca: always install filament library
bundles.insert(PresetBundle::ORCA_FILAMENT_LIBRARY);
for (auto &dir_entry : boost::filesystem::directory_iterator(rsrc_path)) {
const auto &path = dir_entry.path();
std::string file_path = path.string();
if (is_json_file(file_path)) {
const auto path_in_vendor = vendor_path / path.filename();
std::string vendor_name = path.filename().string();
// Remove the .json suffix.
vendor_name.erase(vendor_name.size() - 5);
if (bundles.find(vendor_name) != bundles.end())continue;
// A vendor is named by its profile or, where the build ships preset caches
// instead of the raw profile JSONs, by its cache alone.
for (const std::string &vendor_name : vendor_names_in(rsrc_path)) {
if (bundles.find(vendor_name) != bundles.end())continue;
const auto is_vendor_enabled = (vendor_name == PresetBundle::ORCA_DEFAULT_BUNDLE) // always update configs from resource to vendor for ORCA_DEFAULT_BUNDLE
|| (enabled_vendors.find(vendor_name) != enabled_vendors.end());
if (enabled_config_update) {
if ( fs::exists(path_in_vendor)) {
if (is_vendor_enabled) {
Semver resource_ver = get_version_from_json(file_path);
Semver vendor_ver = get_version_from_json(path_in_vendor.string());
const auto is_vendor_enabled = (vendor_name == PresetBundle::ORCA_DEFAULT_BUNDLE) // always update configs from resource to vendor for ORCA_DEFAULT_BUNDLE
|| (enabled_vendors.find(vendor_name) != enabled_vendors.end());
if (enabled_config_update) {
if (is_vendor_installed(vendor_name)) {
if (is_vendor_enabled) {
// Orca: whichever form of the vendor resources ships at the newer
// version is the one installing lays down, and the one to judge
// what is installed against.
Semver resource_ver = resource_vendor_version(vendor_name);
// Orca: a vendor installed as a preset cache has no profile
// beside it; the version it was installed at is in the cache.
Semver vendor_ver = installed_vendor_version(vendor_name);
if (vendor_ver < resource_ver) {
BOOST_LOG_TRIVIAL(info) << "[Orca Updater]:found vendor " << vendor_name << " newer version "
<< resource_ver.to_string() << " from resource, old version " << vendor_ver.to_string();
bundles.insert(vendor_name);
}
}
else {
//need to be removed because not installed
fs::remove(path_in_vendor);
const auto path_of_vendor = vendor_path / vendor_name;
if (fs::exists(path_of_vendor))
fs::remove_all(path_of_vendor);
if (vendor_ver < resource_ver) {
BOOST_LOG_TRIVIAL(info) << "[Orca Updater]:found vendor " << vendor_name << " newer version "
<< resource_ver.to_string() << " from resource, old version " << vendor_ver.to_string();
bundles.insert(vendor_name);
}
}
else if (is_vendor_enabled) {
bundles.insert(vendor_name);
else {
//need to be removed because not installed
remove_installed_vendor(vendor_name);
}
}
else if (is_vendor_enabled) {
bundles.insert(vendor_name);
}
}
else if (is_vendor_enabled) {
bundles.insert(vendor_name);
}
}
if (bundles.size() > 0) {
@@ -1163,11 +1159,12 @@ Updates PresetUpdater::priv::get_config_updates(const Semver &old_slic3r_version
auto filament_in_cache = (cache_profile_path / vendor_name / PRESET_FILAMENT_NAME);
auto machine_in_cache = (cache_profile_path / vendor_name / PRESET_PRINTER_NAME);
if (( fs::exists(path_in_vendor))
if (is_vendor_installed(vendor_name)
|| fs::exists(print_in_cache)
|| fs::exists(filament_in_cache)
|| fs::exists(machine_in_cache)) {
Semver vendor_ver = get_version_from_json(path_in_vendor.string());
// Orca: a vendor installed as a preset cache carries its version there.
Semver vendor_ver = installed_vendor_version(vendor_name);
std::map<std::string, std::string> key_values;
std::vector<std::string> keys(3);

View File

@@ -17,6 +17,7 @@ add_executable(${_TEST_NAME}_tests
test_preset_bundle_loading.cpp
test_preset_setting_id.cpp
test_preset_diff.cpp
test_vendor_cache.cpp
test_elephant_foot_compensation.cpp
test_fill_plane_path.cpp
test_geometry.cpp

View File

@@ -146,7 +146,7 @@ TEST_CASE("Current vendor type tolerates missing printer model", "[Preset][Bundl
{
PresetBundle bundle;
VendorProfile orca_vendor("ORCA");
VendorProfile orca_vendor; orca_vendor.id = "ORCA";
VendorProfile::PrinterModel model;
model.name = "Orca Test";
orca_vendor.models.emplace_back(model);

File diff suppressed because it is too large Load Diff