Compare commits

..

3 Commits

Author SHA1 Message Date
SoftFever
ba5ecdfea9 Cut redundant work and duplication from the preset cache paths
The schema fingerprint, filament library version and cache blob are no
longer recomputed, re-read or copied once per vendor on the startup path,
and the setup wizard's completeness check now sees vendors shipped as
caches alone. Duplicated reset/join/slurp blocks are folded into helpers.
2026-07-29 21:56:17 +08:00
SoftFever
e5aa9a8cfc Speed up profile loading with per-vendor preset caches
Each vendor's system presets are serialized into a <vendor>.opc cache
that the app loads instead of parsing the profile JSONs, falling back
to the parse whenever no cache covers what is installed. Shipped builds
carry the caches instead of the raw profiles, installing and updating
treat a vendor's cache as its installation, and the setup wizard loads
through the same path. Per-platform scripts and CI generate the caches
at build time; tests and a design doc cover the format and its
validation.
2026-07-29 21:56:17 +08:00
Kiss Lorand
29d4513694 Fix overlapping brims (#14991) 2026-07-28 17:46:14 -03:00
46 changed files with 3059 additions and 821 deletions

5
.gitattributes vendored
View File

@@ -1,2 +1,7 @@
# Set the default behavior, in case people don't have core.autocrlf set.
* text=auto
# Shell scripts are run by Git Bash on Windows CI, which cannot read a script
# with CRLF line endings: it fails on the first line. Windows checkouts default
# to core.autocrlf=true, so keep these LF whatever the platform.
*.sh text eol=lf

View File

@@ -162,6 +162,14 @@ jobs:
retention-days: 5
if-no-files-found: error
- name: Build system preset cache (macOS)
if: runner.os == 'macOS' && !inputs.macos-combine-only
working-directory: ${{ github.workspace }}
shell: bash
# The bundle was already packed from resources/, so the caches have to be
# installed into it here; the source tree keeps its JSONs for later jobs.
run: ./scripts/build_preset_cache.sh -b build/${{ inputs.arch }} build/${{ inputs.arch }}/OrcaSlicer/OrcaSlicer.app/Contents/Resources/profiles
- name: Pack macOS app bundle ${{ inputs.arch }}
if: runner.os == 'macOS' && !inputs.macos-combine-only
working-directory: ${{ github.workspace }}
@@ -390,6 +398,13 @@ jobs:
if ($arch -eq "arm64") { .\build_release_vs.bat slicer arm64 tests } else { .\build_release_vs.bat slicer tests }
shell: pwsh
- name: Build system preset cache (Windows)
if: runner.os == 'Windows'
shell: cmd
# Shipped into both the already-installed tree (portable zip, MSIX) and
# the checkout cpack re-installs from when it builds the NSIS installer.
run: scripts\build_preset_cache.bat "%BUILD_DIR%" "resources\profiles" "%BUILD_DIR%\OrcaSlicer\resources\profiles"
- name: Pack unit tests Win
if: runner.os == 'Windows'
working-directory: ${{ github.workspace }}
@@ -539,6 +554,20 @@ jobs:
retention-days: 5
if-no-files-found: error
- name: Build system preset cache (Linux)
if: runner.os == 'Linux'
shell: bash
run: |
# Both were packed from resources/ before the caches existed, so the
# AppImage is unpacked first and the caches shipped into it and into
# the package tree; the source tree keeps its JSONs for later steps.
appimage=$(find build -maxdepth 1 -name "OrcaSlicer_Linux_AppImage*.AppImage" | head -1)
chmod +x "$appimage"
"$appimage" --appimage-extract
./scripts/build_preset_cache.sh -b build build/package/resources/profiles squashfs-root/resources/profiles
appimagetool=$(find build -name "appimagetool.AppImage" | head -1)
ARCH=$(uname -m) "$appimagetool" --appimage-extract-and-run squashfs-root "$appimage"
rm -rf squashfs-root
# Ship the freshly-built validator so slice_check_linux (build_all.yml)
# can slice-sweep the shipped profiles with this PR's engine. Taken from
# the aarch64 leg so the sweep also exercises the arm build; x86_64 on

1
.gitignore vendored
View File

@@ -49,3 +49,4 @@ internal_docs/
# Python bytecode
__pycache__/
*.pyc
*.opc

View File

@@ -567,6 +567,8 @@ if [[ -n "${BUILD_ORCA}" ]] || [[ -n "${BUILD_TESTS}" ]] ; then
print_and_run cmake --build $BUILD_DIR --config "${BUILD_CONFIG}" --target OrcaSlicer
echo "Building OrcaSlicer_profile_validator .."
print_and_run cmake --build $BUILD_DIR --config "${BUILD_CONFIG}" --target OrcaSlicer_profile_validator
echo "Building generate_system_cache ..."
print_and_run cmake --build $BUILD_DIR --config "${BUILD_CONFIG}" --target generate_system_cache
./scripts/run_gettext.sh
fi
if [[ -n "${BUILD_TESTS}" ]] ; then

218
docs/HLSD/preset-cache.md Normal file
View File

@@ -0,0 +1,218 @@
# System Preset Cache — High Level Design
## Why it exists
OrcaSlicer ships tens of thousands of system preset JSON files. Every launch used to
parse all of them: read each vendor profile, walk its machine, process and filament
sub-files, resolve inheritance, and build the preset collections from scratch. That
parse dominated startup, and it produced the same result every time, because system
presets only change when the app is updated or a profile update is installed.
The preset cache replaces that parse with a read. Each vendor's fully-resolved presets
are serialized once — at build time, in CI — into a single binary file that the app
loads directly into memory. Nothing is recomputed at startup unless something changed.
The cache is **only ever an optimization**. Every rule below exists to guarantee that a
cache is either provably equivalent to parsing the JSONs, or rejected. There is no
"mostly right" cache.
## The unit is one vendor
A cache covers exactly one vendor. `BBL.opc` sits beside `BBL.json` and holds
everything `BBL.json` and the `BBL/` sub-file tree would have produced.
Per-vendor granularity is what makes the system practical:
- A vendor whose profile is bumped invalidates only its own cache. The other 60-odd
vendors keep theirs.
- The setup wizard, which loads vendors one at a time, gets the same speedup as
startup without a second code path.
- A vendor with no cache, or a broken one, costs only that vendor a parse.
A cache holds *system* presets only. User presets, project settings and modified
presets are never serialized — they have their own storage and their own lifecycle.
## Where the files live
| Location | Contents on a shipped build | Role |
|---|---|---|
| `resources/profiles/` | `<vendor>.opc` alone — the profile and its preset JSONs both pruned | What the app ships with; the fallback everything falls back to |
| `<data_dir>/system/` | `<vendor>.opc` alone, or `<vendor>.json` + `<vendor>/` after an update | What the user has installed |
| `<data_dir>/system/` (dev build) | `<vendor>.json` + `<vendor>/` + `<vendor>.opc` written at runtime | A developer tree caches as it parses |
Two forms of the same vendor therefore exist, and the system's central rule is that
**a vendor's cache is the whole of it**. Where a cache ships or is installed, no profile
and no preset JSONs sit beside it: the cache carries the presets, the vendor profile,
and the version stamp that says which release it came from. A vendor is "installed" if
either form is present, and its installed version is read from whichever form is there.
What stays beside the caches in `resources/profiles/` is everything that is not a
preset: each vendor's directory of printer thumbnails, cover images, bed models and
hotend meshes, which are read from disk by path and were never part of the cache. Files
that are not vendors at all, `blacklist.json` chief among them, are untouched.
The alternative — shipping both and treating the cache as a sidecar — was rejected. It
doubles the installed size, and it creates a class of bug where the two disagree and
the app's behavior depends on which one a given code path happened to read.
## What a cache file is
A fixed-size header followed by one binary stream.
The header carries a magic number, the cache format version, the payload size and a
CRC32 of the payload. It exists so that a truncated download, a half-written file or a
file from an entirely different program is rejected in microseconds, before anything
tries to interpret it.
The payload opens with the stamps that decide whether the cache may be used at all —
format version, schema fingerprint, vendor name, vendor version, filament library
version — and then the vendor's data: the vendor profiles, the five preset collections
(print, SLA print, filament, SLA material, printer), the config and filament-id lookup
maps, the obsolete-preset lists, and the count of errors the original parse hit.
Two deliberate choices in the layout:
- **Stamps come first**, so the question "what version is this vendor installed at?"
can be answered by reading the first kilobyte. The updater asks that question for
every vendor on every launch; reading tens of megabytes to answer it would give back
the startup time the cache saved.
- **Defaults are not stored.** Every collection reconstructs its default presets the
way the JSON path does, and the cache carries only what a parse would have added on
top. This keeps the cache a record of the vendor's data, not a memory image of the
program's state.
## When a cache may be used
A cache is accepted only if every gate below passes. Any failure means "parse the
JSONs instead" — never a hard error, never a partial load.
**1. Integrity.** Magic number, plausible size, CRC32 over the payload.
**2. Cache format version.** A single integer bumped by hand whenever the binary layout
changes in a way nothing else would catch: reordering or retyping a serialized field,
or changing what the cache's own stamps mean.
**3. Schema fingerprint.** A checksum over the app version and the entire print-config
option schema — every option's key, type, wire ordinal and enum values. This is the
gate that makes the cache safe across development: adding a config option, changing its
type, or reordering the enum values of an existing one all change the fingerprint, so
caches from before the change are rejected without anyone having to remember to bump
anything. It also means a cache never crosses app versions.
**4. Vendor identity and version.** The cache names the vendor it holds and the profile
version it was built from. It is accepted only if that version is at least as new as
the profile now on disk. Where no profile sits beside the cache — the shipped,
cache-only form — the comparison is skipped, because nothing on disk can be newer than
a cache that is the installation.
**5. Filament library version.** Every vendor's filaments inherit from the shared Orca
filament library, so a vendor's cache is only valid against the library it was resolved
against. Bumping the library invalidates every vendor's cache, which is correct and
is why the library's version is stamped into all of them.
A vendor profile with no parsable version is never cached and never served from a
cache. There would be no way to tell later whether the cache had gone stale, and a
cache nothing can invalidate is worse than no cache.
## How a vendor is loaded
When the app loads a vendor, it tries, in order:
1. The cache in the directory it was asked to load from — normally `<data_dir>/system/`.
2. The shipped cache in `resources/profiles/`.
3. Parsing the JSONs — from the data directory if the profile is installed there, and
from `resources/profiles/` otherwise, which on a shipped build only has JSONs for a
vendor that has no cache.
The second tier is what makes app upgrades work. After an upgrade, a cache the previous
version installed fails the fingerprint gate; the new build's own shipped cache answers
instead, and the user never sees a parse. The stale installed file is simply ignored
until the next profile update overwrites it.
If a parse does happen and the vendor's profile carries a version, the app writes the
cache back beside where it looked for the vendor. That is how a developer build warms
itself up on second launch, and how a vendor delivered by a profile update becomes
cached without waiting for the next release.
## How a vendor is installed
Installing copies from `resources/profiles/` into `<data_dir>/system/`. A shipped build
offers only a cache and a source tree only JSONs, but a partially-generated tree can
have both, at different versions, so the installer picks the form that ships at the
**newer version** and installs only that one:
- Cache newer or equal, and readable → copy the `.opc`, and delete any profile and
vendor directory a previous install left behind, so nothing can shadow it.
- Profile newer, or the cache unreadable or absent → copy the profile and the vendor's
preset JSONs exactly as the app did before caches existed, and delete any stale `.opc`.
The result is that only one form of a vendor is ever present, and it is the newest one
the build has. This matters most for the update check, which compares what is installed
against what installing *would* lay down: if those two disagreed about which form
counts, a vendor could reinstall on every launch forever, or silently never update.
Profile updates delivered over the air always arrive as JSONs, and they win — an
updated vendor's real profile lands in the data directory, the shipped cache is older
and gets rejected, and the vendor is parsed and re-cached.
## How the caches are produced
Cache generation is a build step, not something a user ever runs.
One script per platform does the whole job, and CI calls it once on each. It builds a
small dev-utility that loads a profiles directory exactly as the app would, with cache
writing enabled, dropping a `<vendor>.opc` beside every vendor profile it parses; then
it copies those caches into each packaged application it was pointed at and deletes
every preset JSON they replace — the vendor's own profile included. Only a vendor that
actually has a cache is pruned, so a vendor the generator skipped keeps its JSONs and is
simply parsed at startup.
Because the schema fingerprint includes the app version, caches must be generated by
the same build that ships them. Generation runs after the build, in the same job.
## Behavior when things go wrong
The system is designed so that no cache problem is fatal:
- **Corrupt, truncated or foreign file** — rejected at the header, vendor parsed.
- **Cache from another app version or schema** — rejected at the fingerprint, vendor
parsed or served from the shipped cache.
- **Stale cache** — rejected on the version stamps, vendor parsed and re-cached.
- **Failure part-way through reading** — the bundle is reset to a clean state before
falling back, so a half-loaded cache can never leak into the parsed result.
- **A vendor that can be neither read nor parsed** — logged, and left out. The setup
wizard drops that vendor from its list and opens with the rest; startup records the
error alongside the vendors that did load. One broken vendor never takes the app down.
The one genuine limit: on a shipped build a vendor is its cache and nothing else, so a
rejected cache has nothing to fall back to for that vendor. This is by design — the
alternative is shipping every preset twice — and it is why the acceptance gates are
conservative and why CI generates the caches with the same build that ships them. The
recovery path is a profile update, which delivers real JSONs.
It also means nothing may quietly assume a `<vendor>.json` exists. Discovery, version
checks and the update decision all read whichever form is present, and a code path that
enumerates only `*.json` will find no vendors at all in a packaged build.
## Maintenance rules
- **Adding or changing a config option** needs nothing. The fingerprint covers it.
- **Changing what a cache serializes**, or the order it serializes it in, requires
bumping the cache format version by hand.
- **Bumping a vendor profile's version** invalidates that vendor's cache and nothing
else. Bumping the filament library invalidates all of them.
- **Caches are never committed.** They are build artifacts, generated per build,
ignored by git.
## Where this lives in the tree
| Area | Files |
|---|---|
| Cache format, read/write, load and save | `src/libslic3r/PresetBundle.{hpp,cpp}` |
| Per-preset serialization | `src/libslic3r/Preset.{hpp,cpp}` |
| Vendor discovery, installed/shipped versions, installation | `src/libslic3r/PresetBundle.cpp` |
| Update and reinstall decisions | `src/slic3r/Utils/PresetUpdater.cpp` |
| Setup wizard and printer-selection dialog | `src/slic3r/GUI/ConfigWizard.cpp`, `src/slic3r/GUI/WebGuideDialog.cpp` |
| Generator tool | `src/dev-utils/generate_system_cache.cpp` |
| Build and packaging script | `scripts/build_preset_cache.{sh,bat}` |
| Tests | `tests/libslic3r/test_vendor_cache.cpp` |

View File

@@ -0,0 +1,122 @@
@echo off
rem Build the per-vendor system preset caches (one <vendor>.opc per vendor) by
rem running the generate_system_cache.exe dev tool against a profiles directory,
rem and make every profiles directory named on the command line ship-ready:
rem install the caches into it and delete the preset JSONs they replace, so a
rem build ships one copy of its presets instead of two.
rem
rem scripts\build_preset_cache.bat [build_dir] [target_dir ...]
rem
rem build_dir defaults to "build"
rem target_dir profiles directories to ship into. Caches are generated into
rem the source tree's resources\profiles, which is what every
rem packaging step copies from; a target may be that same
rem directory, which then only gets pruned.
rem
rem Shipping deletes, so it is a CI packaging step. A vendor's own <vendor>.json
rem goes along with its preset JSONs: the cache carries the vendor profile and
rem the version it was built at, so discovery, version checks and installing all
rem read it there. Only a vendor that has a cache is pruned, so non-vendor JSONs
rem (blacklist.json) are left alone, as are the vendor directories themselves -
rem thumbnails, covers and bed models still live there.
rem
rem set CONFIG=<cfg> to pin the build config for multi-config generators
rem (default: the config of the tool already in the build tree, else Release)
setlocal enabledelayedexpansion
set "REPO_ROOT=%~dp0.."
set "BUILD_DIR=%~1"
if "%BUILD_DIR%"=="" set "BUILD_DIR=build"
if not exist "%BUILD_DIR%\" (
echo ERROR: build tree not found: %BUILD_DIR% 1>&2
exit /b 1
)
if not "%~1"=="" shift
rem Newest match wins: a stale binary silently produces a stale cache layout.
call :find_tool
if not defined CONFIG (
for %%c in (Debug Release RelWithDebInfo MinSizeRel) do (
echo !TOOL! | findstr /i "\\%%c\\" >nul && set "CONFIG=%%c"
)
)
if not defined CONFIG set "CONFIG=Release"
echo Building generate_system_cache in %BUILD_DIR% (%CONFIG%)
cmake --build "%BUILD_DIR%" --config %CONFIG% --target generate_system_cache
if errorlevel 1 (
echo ERROR: could not build generate_system_cache - configure the build tree with -DORCA_TOOLS=ON: 1>&2
echo cmake -S "%REPO_ROOT%" -B "%BUILD_DIR%" -DORCA_TOOLS=ON 1>&2
exit /b 1
)
call :find_tool
if not defined TOOL (
echo ERROR: generate_system_cache.exe not found under %BUILD_DIR% - build with -DORCA_TOOLS=ON 1>&2
exit /b 1
)
set "PROFILES=%REPO_ROOT%\resources\profiles"
if not exist "%PROFILES%\" (
echo ERROR: profiles directory not found: %PROFILES% 1>&2
exit /b 1
)
for %%d in ("%PROFILES%") do set "PROFILES=%%~fd"
rem Add the slicer's runtime DLL directory to PATH so generate_system_cache.exe
rem can resolve its dependencies (TKernel.dll etc.) without a full install step.
set "DLL_DIR="
for /f "delims=" %%f in ('dir /s /b "%BUILD_DIR%\TKernel.dll" 2^>nul') do (
if not defined DLL_DIR set "DLL_DIR=%%~dpf"
)
if defined DLL_DIR set "PATH=%DLL_DIR%;%PATH%"
echo Generating per-vendor preset caches in %PROFILES%
rem Start clean so vendors that went away - and caches written by older tool
rem versions - don't linger next to the freshly generated ones.
del /q "%PROFILES%\*.opc" 2>nul
del /q "%PROFILES%\*.cache" 2>nul
"%TOOL%" --path "%PROFILES%" --log_level 2
if errorlevel 1 exit /b %errorlevel%
:next_target
if "%~1"=="" exit /b 0
call :ship "%~1"
if errorlevel 1 exit /b 1
shift
goto :next_target
:ship
set "TARGET=%~1"
if not exist "%TARGET%\" (
echo ERROR: profiles directory not found: %TARGET% 1>&2
exit /b 1
)
for %%d in ("%TARGET%") do set "TARGET=%%~fd"
if /i not "%TARGET%"=="%PROFILES%" copy /y "%PROFILES%\*.opc" "%TARGET%\" >nul
set /a SHIPPED=0
set /a PRUNED=0
for %%c in ("%PROFILES%\*.opc") do (
set /a SHIPPED+=1
set "VENDOR=%%~nc"
if exist "%TARGET%\!VENDOR!.json" (
del /q "%TARGET%\!VENDOR!.json"
set /a PRUNED+=1
)
if exist "%TARGET%\!VENDOR!\" (
for /f %%n in ('dir /s /b "%TARGET%\!VENDOR!\*.json" 2^>nul ^| find /c /v ""') do set /a PRUNED+=%%n
del /s /q "%TARGET%\!VENDOR!\*.json" >nul 2>&1
rem Deepest first, so a directory the delete above emptied goes too; rd
rem refuses the ones still holding covers or meshes.
for /f "delims=" %%d in ('dir /s /b /ad "%TARGET%\!VENDOR!" 2^>nul ^| sort /r') do rd "%%d" 2>nul
)
)
echo %TARGET%: !SHIPPED! caches, dropped !PRUNED! preset JSONs
exit /b 0
:find_tool
set "TOOL="
for /f "delims=" %%f in ('dir /s /b /o-d "%BUILD_DIR%\generate_system_cache.exe" 2^>nul') do (
if not defined TOOL set "TOOL=%%f"
)
exit /b 0

143
scripts/build_preset_cache.sh Executable file
View File

@@ -0,0 +1,143 @@
#!/usr/bin/env bash
# Build the per-vendor system preset caches (one <vendor>.opc per vendor) by
# running the generate_system_cache dev tool against a profiles directory, and
# make every profiles directory named on the command line ship-ready: install
# the caches into it and delete the preset JSONs they replace, so a build ships
# one copy of its presets instead of two.
#
# ./scripts/build_preset_cache.sh # caches into resources/profiles
# ./scripts/build_preset_cache.sh -b build/arm64 # search this build tree for the tool
# ./scripts/build_preset_cache.sh <dir> [<dir> ...] # and ship into these profiles dirs
#
# Caches are generated into the source tree's resources/profiles, which is what
# every packaging step copies from. Shipping deletes, so it is a CI packaging
# step: pass packaged output directories, or the checkout of a build that is
# about to be packaged from it.
#
# A vendor's own <vendor>.json goes along with its preset JSONs: the cache
# carries the vendor profile and the version it was built at, so discovery,
# version checks and installing all read it there. A shipped vendor is its cache
# and nothing else. Only a vendor that has a cache is pruned, so an ungenerated
# vendor keeps its JSONs and is simply parsed at startup; non-vendor JSONs
# (blacklist.json) are left alone, as are the vendor directories themselves —
# thumbnails, covers and bed models still live there.
#
# -b <dir> build tree holding the tool
# (default: build/arm64, build/x86_64, or build — first that exists)
# -p <dir> profiles directory to generate caches into
# (default: <repo>/resources/profiles)
# -c <cfg> build config for multi-config generators
# (default: the config of the tool already in the build tree, else
# the build tree's CMAKE_BUILD_TYPE)
# -n skip the rebuild and run the tool already in the build tree
# -l <level> tool log level (default: 2)
set -euo pipefail
repo_root="$(cd "$(dirname "$0")/.." && pwd -P)"
build_dir=""
profiles_dir=""
config=""
build_tool=1
log_level=2
while getopts "b:p:c:l:nh" opt; do
case $opt in
b) build_dir="$OPTARG" ;;
p) profiles_dir="$OPTARG" ;;
c) config="$OPTARG" ;;
n) build_tool=0 ;;
l) log_level="$OPTARG" ;;
h) sed -n '2,${/^#/!q;p;}' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
*) exit 1 ;;
esac
done
shift $((OPTIND - 1))
if [ -z "$build_dir" ]; then
for candidate in "$repo_root/build/arm64" "$repo_root/build/x86_64" "$repo_root/build"; do
if [ -d "$candidate" ]; then build_dir="$candidate"; break; fi
done
fi
if [ -z "$build_dir" ] || [ ! -d "$build_dir" ]; then
echo "ERROR: build tree not found (pass -b <build_dir>)" >&2
exit 1
fi
# Newest match wins: multi-config trees keep one binary per config, and a stale
# one silently produces a stale cache layout.
find_tool() {
local best="" f
while IFS= read -r f; do
[ -n "$f" ] || continue
if [ -z "$best" ] || [ "$f" -nt "$best" ]; then best="$f"; fi
done < <(find "$build_dir" -name generate_system_cache -type f 2>/dev/null)
printf '%s' "$best"
}
tool=$(find_tool)
if [ -z "$config" ]; then
case "$tool" in
*/Debug/*) config=Debug ;;
*/Release/*) config=Release ;;
*/RelWithDebInfo/*) config=RelWithDebInfo ;;
*/MinSizeRel/*) config=MinSizeRel ;;
*) config=$(sed -n 's/^CMAKE_BUILD_TYPE:[A-Z]*=\(.\+\)$/\1/p' "$build_dir/CMakeCache.txt" 2>/dev/null | head -1 || true) ;;
esac
fi
if [ "$build_tool" = 1 ]; then
echo "Building generate_system_cache in $build_dir${config:+ ($config)}"
build_args=(--build "$build_dir" --target generate_system_cache)
if [ -n "$config" ]; then build_args+=(--config "$config"); fi
if ! cmake "${build_args[@]}"; then
echo "ERROR: could not build generate_system_cache — configure the build tree with -DORCA_TOOLS=ON:" >&2
echo " cmake -S \"$repo_root\" -B \"$build_dir\" -DORCA_TOOLS=ON" >&2
exit 1
fi
tool=$(find_tool)
fi
if [ -z "$tool" ]; then
echo "ERROR: generate_system_cache not found under $build_dir — build with -DORCA_TOOLS=ON" >&2
exit 1
fi
if [ -z "$profiles_dir" ]; then profiles_dir="$repo_root/resources/profiles"; fi
if [ ! -d "$profiles_dir" ]; then
echo "ERROR: profiles directory not found: $profiles_dir" >&2
exit 1
fi
profiles_dir=$(cd "$profiles_dir" && pwd -P)
# Start clean so vendors that went away — and caches written by older tool
# versions — don't linger next to the freshly generated ones.
echo "Generating per-vendor preset caches in $profiles_dir"
rm -f "$profiles_dir"/*.opc "$profiles_dir"/*.cache
"$tool" --path "$profiles_dir" --log_level "$log_level"
for target in "$@"; do
resolved=$(cd "$target" 2>/dev/null && pwd -P) || {
echo "ERROR: profiles directory not found: $target" >&2
exit 1
}
if [ "$resolved" != "$profiles_dir" ]; then
cp "$profiles_dir"/*.opc "$resolved"/
fi
pruned=0
shipped=0
for cache in "$profiles_dir"/*.opc; do
vendor=$(basename "$cache" .opc)
shipped=$(( shipped + 1 ))
if [ -f "$resolved/$vendor.json" ]; then
rm -f "$resolved/$vendor.json"
pruned=$(( pruned + 1 ))
fi
[ -d "$resolved/$vendor" ] || continue
n=$(find "$resolved/$vendor" -name '*.json' | wc -l)
find "$resolved/$vendor" -name '*.json' -delete
find "$resolved/$vendor" -type d -empty -delete
pruned=$(( pruned + n ))
done
echo "$resolved: $shipped caches, dropped $pruned preset JSONs"
done

View File

@@ -20,6 +20,16 @@ if (SLIC3R_ENC_CHECK)
)
endif()
if (ORCA_TOOLS)
set(_DEV_DEFS -DBOOST_ALL_NO_LIB -DBOOST_USE_WINAPI_VERSION=0x602 -DBOOST_SYSTEM_USE_UTF8)
# generate_system_cache: pre-generates per-vendor <vendor>.opc files under resources/profiles for CI bundling.
add_executable(generate_system_cache generate_system_cache.cpp)
target_link_libraries(generate_system_cache libslic3r boost_headeronly)
target_compile_definitions(generate_system_cache PRIVATE ${_DEV_DEFS})
endif()
# Function that adds source file encoding check to a target
# using the above encoding-check binary

View File

@@ -0,0 +1,84 @@
#include "libslic3r/PresetBundle.hpp"
#include "libslic3r/Preset.hpp"
#include "libslic3r/Utils.hpp"
#include <boost/algorithm/string/predicate.hpp>
#include <boost/filesystem.hpp>
#include <boost/log/trivial.hpp>
#include <boost/program_options.hpp>
#include <iostream>
using namespace Slic3r;
namespace fs = boost::filesystem;
namespace po = boost::program_options;
int main(int argc, char* argv[])
{
po::options_description desc("OrcaSlicer System Cache Generator\nUsage");
// clang-format off
desc.add_options()
("help,h", "Show help")
#ifdef __APPLE__
("path,p", po::value<std::string>()->default_value("../../../../../../../resources/profiles"), "Path to profiles directory")
#else
("path,p", po::value<std::string>()->default_value("../../../resources/profiles"), "Path to profiles directory")
#endif
("log_level,l", po::value<int>()->default_value(2), "Log level (0=trace, 2=info, 4=error)");
// clang-format on
po::variables_map vm;
try {
po::store(po::parse_command_line(argc, argv, desc), vm);
if (vm.count("help")) { std::cout << desc << "\n"; return 0; }
po::notify(vm);
} catch (const po::error& e) {
std::cerr << "Error: " << e.what() << "\n" << desc << "\n";
return 1;
}
const std::string profiles_path = vm["path"].as<std::string>();
const int log_level = vm["log_level"].as<int>();
if (!fs::exists(profiles_path) || !fs::is_directory(profiles_path)) {
std::cerr << "Error: '" << profiles_path << "' is not a valid directory\n";
return 1;
}
set_logging_level(log_level);
set_data_dir(profiles_path);
set_resources_dir(fs::path(profiles_path).parent_path().make_preferred().string());
const fs::path user_dir = fs::path(data_dir()) / PRESET_USER_DIR;
if (!fs::exists(user_dir))
fs::create_directories(user_dir);
AppConfig app_config;
app_config.set("preset_folder", "default");
auto preset_bundle = std::make_unique<PresetBundle>();
preset_bundle->set_is_validation_mode(true);
preset_bundle->set_default_suppressed(true);
preset_bundle->set_generate_vendor_caches(true);
std::cout << "Loading system presets from: " << profiles_path << "\n";
try {
// In validation mode data_dir() is the profiles directory set above, so the
// loader writes each <vendor>.opc next to its <vendor>.json as it parses it.
preset_bundle->load_presets(app_config, ForwardCompatibilitySubstitutionRule::EnableSilent);
} catch (const std::exception& ex) {
std::cerr << "Failed to load presets: " << ex.what() << "\n";
return 1;
}
size_t cache_count = 0;
for (auto& entry : fs::directory_iterator(profiles_path))
if (boost::iends_with(entry.path().string(), ".opc"))
++ cache_count;
if (cache_count == 0) {
std::cerr << "No vendor cache files were generated under " << profiles_path << "\n";
return 1;
}
std::cout << "Generated " << cache_count << " vendor cache file(s) under " << profiles_path << "\n";
return 0;
}

View File

@@ -32,15 +32,13 @@ static void append_and_translate(ExPolygons &dst, const ExPolygons &src, const P
for (; dst_idx < dst.size(); ++dst_idx)
dst[dst_idx].translate(instance_shift);
}
// BBS: generate brim area by objs
static void append_and_translate(ExPolygons& dst, const ExPolygons& src,
const PrintInstance& instance, size_t instance_idx, std::map<ObjectInstanceID, ExPolygons>& brimAreaMap) {
// Orca: Translate the brim area into print coordinates and store it per instance.
static void append_and_translate(const ExPolygons& src, const PrintInstance& instance,
size_t instance_idx, std::map<ObjectInstanceID, ExPolygons>& brimAreaMap) {
ExPolygons srcShifted = src;
Point instance_shift = instance.shift_without_plate_offset();
for (size_t src_idx = 0; src_idx < srcShifted.size(); ++src_idx)
srcShifted[src_idx].translate(instance_shift);
srcShifted = diff_ex(srcShifted, dst);
//expolygons_append(dst, temp2);
for (ExPolygon& expoly : srcShifted)
expoly.translate(instance_shift);
expolygons_append(brimAreaMap[{ instance.print_object->id(), instance_idx }], std::move(srcShifted));
}
@@ -572,7 +570,7 @@ static ExPolygons outer_inner_brim_area(const Print& print,
for (size_t instance_idx = 0; instance_idx < object->instances().size(); ++instance_idx) {
const PrintInstance& instance = object->instances()[instance_idx];
if (!brim_area_object.empty())
append_and_translate(brim_area, brim_area_object, instance, instance_idx, brimAreaMap);
append_and_translate(brim_area_object, instance, instance_idx, brimAreaMap);
append_and_translate(no_brim_area, no_brim_area_object, instance);
append_and_translate(holes, holes_object, instance);
append_and_translate(objectIslands, objectIsland, instance);
@@ -875,6 +873,14 @@ void make_brim(const Print& print, PrintTryCancel try_cancel, Polygons& islands_
ExPolygons islands_area_ex = outer_inner_brim_area(print,
float(flow.scaled_spacing()), brimAreaMap, objPrintVec, printExtruders);
if (!print.config().combine_brims) {
ExPolygons claimed_area;
for (auto& [_, areas] : brimAreaMap) {
areas = diff_ex(areas, claimed_area);
expolygons_append(claimed_area, areas);
}
}
// BBS: Find boundingbox of the first layer
for (const ObjectID printObjID : print.print_object_ids()) {
BoundingBox bbx;

View File

@@ -27,6 +27,9 @@
#include <cereal/access.hpp>
#include <cereal/types/base_class.hpp>
// The serialize() members below archive ConfigOption hierarchies through
// cereal::base_class, whose registration machinery lives in polymorphic.hpp.
#include <cereal/types/polymorphic.hpp>
namespace Slic3r {
struct FloatOrPercent

View File

@@ -147,6 +147,17 @@ Semver get_version_from_json(std::string file_path)
return Semver();
//throw ConfigurationError(format("Failed loading configuration file \"%1%\": %2%", file_path, err.what()));
}
catch(...) {
return Semver();
}
}
std::string get_vendor_cache_version(const std::string& json_path)
{
// The version a vendor's cache is stamped with. A profile without a parsable
// version cannot be judged for staleness, so it is simply never cached.
const Semver ver = get_version_from_json(json_path);
return ver.valid() ? ver.to_string() : std::string();
}
//BBS: add a function to load the key-values from xxx.json

View File

@@ -16,6 +16,8 @@
#include "Semver.hpp"
#include "ProjectTask.hpp"
#include <cereal/access.hpp>
//BBS: change system directories
#define PRESET_SYSTEM_DIR "system"
#define PRESET_USER_DIR "user"
@@ -114,6 +116,10 @@ extern Semver get_version_from_json(std::string file_path);
//BBS: add a function to load the key-values from xxx.json
extern int get_values_from_json(std::string file_path, std::vector<std::string>& keys, std::map<std::string, std::string>& key_values);
// Returns the version a vendor JSON's preset cache is stamped with: its Semver
// string, or an empty string when the profile carries no usable version.
extern std::string get_vendor_cache_version(const std::string& json_path);
extern ConfigFileType guess_config_file_type(const boost::property_tree::ptree &tree);
extern void extend_default_config_length(DynamicPrintConfig& config, const bool set_nil_to_default, const DynamicPrintConfig& defaults);
@@ -131,6 +137,10 @@ public:
PrinterVariant() {}
PrinterVariant(const std::string &name) : name(name) {}
std::string name;
// All fields, declaration order — keep in sync; bump CACHE_VERSION on change.
template<class Archive>
void serialize(Archive& ar) { ar(name); } // PrinterVariant
};
struct PrinterModel {
@@ -139,7 +149,7 @@ public:
std::string name;
//BBS: this is internal id for the printer. Currently only used for searching in database
std::string model_id;
PrinterTechnology technology;
PrinterTechnology technology = ptFFF;
std::string family;
std::vector<PrinterVariant> variants;
std::vector<std::string> default_materials;
@@ -162,6 +172,17 @@ public:
}
const PrinterVariant* variant(const std::string &name) const { return const_cast<PrinterModel*>(this)->variant(name); }
// All fields, declaration order — keep in sync; bump CACHE_VERSION on change.
template<class Archive>
void serialize(Archive& ar) // PrinterModel
{
ar(id, name, model_id, technology, family, variants, default_materials,
not_support_bed_types, bed_model, bed_texture, image_bed_type,
bottom_texture_end_name, use_double_extruder_default_texture,
bottom_texture_rect, bottom_texture_rect_longer, middle_texture_rect,
hotend_model);
}
};
std::vector<PrinterModel> models;
@@ -173,6 +194,14 @@ public:
bool valid() const { return ! name.empty() && ! id.empty() && config_version.valid(); }
// All fields, declaration order — keep in sync; bump CACHE_VERSION on change.
template<class Archive>
void serialize(Archive& ar) // VendorProfile
{
ar(name, id, config_version, config_update_url, changelog_url,
models, default_filaments, default_sla_materials);
}
// Load VendorProfile from an ini file.
// If `load_all` is false, only the header with basic info (name, version, URLs) is loaded.
static VendorProfile from_ini(const boost::filesystem::path &path, bool load_all=true);
@@ -425,12 +454,30 @@ public:
// BBS: move constructor to public
Preset(Type type, const std::string &name, bool is_default = false) : type(type), is_default(is_default), name(name) {}
protected:
// Default constructor is public so cereal can default-construct elements when
// deserializing std::vector<Preset> (std::allocator is not a cereal::access friend).
Preset() = default;
protected:
friend class PresetCollection;
friend class PresetBundle;
friend class cereal::access;
// Hand-written cereal serialization for the per-vendor binary cache.
// Lists every data member except the two raw pointers:
// - loading_substitutions: transient parse state, never cached
// - vendor: re-pointed on load from the vendor id stored alongside each preset
// Keep this list in sync with the member declarations, in declaration order;
// bump CACHE_VERSION in PresetBundle.cpp when it changes.
template<class Archive>
void serialize(Archive& ar)
{
ar(type, is_default, is_external, is_system, is_visible, is_dirty,
is_compatible, is_project_embedded, name, file, loaded, config,
alias, renamed_from, m_excluded_from, m_from_orca_filament_lib,
bundle_id, version, ini_str, setting_id, filament_id, user_id,
base_id, sync_info, description, updated_time, key_values);
}
};
bool is_compatible_with_print (const PresetWithVendorProfile &preset, const PresetWithVendorProfile &active_print, const PresetWithVendorProfile &active_printer);

View File

@@ -1,7 +1,18 @@
#include <cassert>
#include <chrono>
#include <ctime>
#include <sstream>
#include "PresetBundle.hpp"
#include <boost/crc.hpp>
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/stream.hpp>
#include <cereal/archives/binary.hpp>
#include <cereal/types/map.hpp>
#include <cereal/types/set.hpp>
#include <cereal/types/string.hpp>
#include <cereal/types/vector.hpp>
#include "PrintConfig.hpp"
#include "libslic3r.h"
#include "I18N.hpp"
@@ -564,6 +575,8 @@ PresetsConfigSubstitutions PresetBundle::load_presets(AppConfig &config, Forward
//BBS: add config related logs
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << boost::format(" enter, substitution_rule %1%, preferred printer_model_id %2%")%substitution_rule%preferred_selection.printer_model_id;
const auto startup_t0 = std::chrono::steady_clock::now();
//BBS: change system config to json
std::tie(substitutions, errors_cummulative) = this->load_system_presets_from_json(substitution_rule);
@@ -589,6 +602,12 @@ PresetsConfigSubstitutions PresetBundle::load_presets(AppConfig &config, Forward
set_calibrate_printer("");
{
const auto total_ms = std::chrono::duration_cast<std::chrono::milliseconds>(
std::chrono::steady_clock::now() - startup_t0).count();
BOOST_LOG_TRIVIAL(info) << "PresetBundle: all presets loaded in " << total_ms << " ms";
}
//BBS: add config related logs
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << boost::format(" finished, returned substitutions %1%")%substitutions.size();
return substitutions;
@@ -1001,6 +1020,8 @@ PresetsConfigSubstitutions PresetBundle::load_user_presets(std::string user, For
bundles.m_bundles.clear();
bundles.WriteUnlock();
const auto user_load_t0 = std::chrono::steady_clock::now();
// Load bundle metadata from _local directory first
fs::path local_dir(folder / PRESET_LOCAL_DIR);
if (fs::exists(local_dir)) {
@@ -1019,7 +1040,6 @@ PresetsConfigSubstitutions PresetBundle::load_user_presets(std::string user, For
metadata.filament_presets.clear();
metadata.printer_presets.clear();
// Add the profiles
this->prints.load_presets(bundle_dir, PRESET_PRINT_NAME, substitutions, substitution_rule, [&](Preset& preset) {
metadata.print_presets.push_back(preset.name);
}, PresetOrigin(PresetOrigin::Kind::LocalBundle, metadata.id));
@@ -1056,7 +1076,6 @@ PresetsConfigSubstitutions PresetBundle::load_user_presets(std::string user, For
metadata.printer_presets.clear();
metadata.is_subscribed = true;
// Load presets from bundle (same logic as __local__)
this->prints.load_presets(bundle_dir, PRESET_PRINT_NAME, substitutions, substitution_rule, [&](Preset& preset) {
metadata.print_presets.push_back(preset.name);
}, PresetOrigin(PresetOrigin::Kind::SubscribedBundle, metadata.id));
@@ -1077,34 +1096,41 @@ PresetsConfigSubstitutions PresetBundle::load_user_presets(std::string user, For
}
}
// BBS do not load sla_print
// BBS: change directoties by design
try {
std::string print_selected_preset_name = prints.get_selected_preset().name;
this->prints.load_presets(dir_user_presets, PRESET_PRINT_NAME, substitutions, substitution_rule);
prints.select_preset_by_name(print_selected_preset_name, false);
} catch (const std::runtime_error &err) {
errors_cummulative += err.what();
// BBS: change directories by design
{
const auto json_t0 = std::chrono::steady_clock::now();
try {
std::string sel = prints.get_selected_preset().name;
this->prints.load_presets(dir_user_presets, PRESET_PRINT_NAME, substitutions, substitution_rule);
prints.select_preset_by_name(sel, false);
} catch (const std::runtime_error& err) { errors_cummulative += err.what(); }
try {
std::string sel = filaments.get_selected_preset().name;
this->filaments.load_presets(dir_user_presets, PRESET_FILAMENT_NAME, substitutions, substitution_rule);
filaments.select_preset_by_name(sel, false);
} catch (const std::runtime_error& err) { errors_cummulative += err.what(); }
try {
std::string sel = printers.get_selected_preset().name;
this->printers.load_presets(dir_user_presets, PRESET_PRINTER_NAME, substitutions, substitution_rule);
printers.select_preset_by_name(sel, false);
} catch (const std::runtime_error& err) { errors_cummulative += err.what(); }
if (!errors_cummulative.empty()) throw Slic3r::RuntimeError(errors_cummulative);
const auto json_ms = std::chrono::duration_cast<std::chrono::milliseconds>(
std::chrono::steady_clock::now() - json_t0).count();
BOOST_LOG_TRIVIAL(info) << "PresetBundle: user presets loaded from JSON in " << json_ms << " ms";
}
try {
std::string filament_selected_preset_name = filaments.get_selected_preset().name;
this->filaments.load_presets(dir_user_presets, PRESET_FILAMENT_NAME, substitutions, substitution_rule);
filaments.select_preset_by_name(filament_selected_preset_name, false);
} catch (const std::runtime_error &err) {
errors_cummulative += err.what();
{
const auto ms = std::chrono::duration_cast<std::chrono::milliseconds>(
std::chrono::steady_clock::now() - user_load_t0).count();
BOOST_LOG_TRIVIAL(info) << "PresetBundle: user + bundle presets loaded in " << ms << " ms";
}
try {
std::string printer_selected_preset_name = printers.get_selected_preset().name;
this->printers.load_presets(dir_user_presets, PRESET_PRINTER_NAME, substitutions, substitution_rule);
printers.select_preset_by_name(printer_selected_preset_name, false);
} catch (const std::runtime_error &err) {
errors_cummulative += err.what();
}
if (!errors_cummulative.empty()) throw Slic3r::RuntimeError(errors_cummulative);
this->update_multi_material_filament_presets();
this->update_compatible(PresetSelectCompatibleType::Never);
set_calibrate_printer("");
return PresetsConfigSubstitutions();
@@ -1210,13 +1236,10 @@ bool PresetBundle::apply_vendor_config(
: std::map<std::string, std::string>();
// Find vendors that need installation
const auto vendor_dir = (fs::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR).make_preferred();
std::vector<std::string> install_bundles;
for (const auto &it : new_vendors) {
if (it.second.size() > 0) {
auto vendor_file = vendor_dir / (it.first + ".json");
if (!fs::exists(vendor_file)) {
if (!is_vendor_installed(it.first)) {
install_bundles.emplace_back(it.first);
}
}
@@ -2225,6 +2248,186 @@ void PresetBundle::remove_users_preset(AppConfig &config, std::map<std::string,
}
// The version of the filament library in effect: the installed profile, the
// installed cache that stands in for one, or — with nothing installed yet — the
// profile that ships. A vendor cache embeds filaments resolved against the
// library, so it is judged against this and never left unconstrained; a cache
// built against an older library would otherwise slip through unnoticed.
static std::string effective_lib_version(const boost::filesystem::path& installed_dir)
{
const std::string lib(PresetBundle::ORCA_FILAMENT_LIBRARY);
// The library ships as a cache like every other vendor, so neither directory
// is guaranteed to hold its profile; ask each for whichever form it has.
for (const boost::filesystem::path& dir : {installed_dir, boost::filesystem::path(resources_dir()) / "profiles"}) {
if (boost::filesystem::exists(dir / (lib + ".json")))
return get_vendor_cache_version((dir / (lib + ".json")).string());
const std::string stamped = PresetBundle::peek_vendor_cache_version((dir / (lib + ".opc")).string(), lib);
if (! stamped.empty())
return stamped;
}
return {};
}
bool is_vendor_installed(const std::string& vendor)
{
const boost::filesystem::path dir = boost::filesystem::path(data_dir()) / PRESET_SYSTEM_DIR;
return boost::filesystem::exists(dir / (vendor + ".json")) || boost::filesystem::exists(dir / (vendor + ".opc"));
}
Semver installed_vendor_version(const std::string& vendor)
{
const boost::filesystem::path dir = boost::filesystem::path(data_dir()) / PRESET_SYSTEM_DIR;
const boost::filesystem::path json = dir / (vendor + ".json");
if (boost::filesystem::exists(json))
return get_version_from_json(json.string());
const auto ver = Semver::parse(PresetBundle::peek_vendor_cache_version((dir / (vendor + ".opc")).string(), vendor));
return ver ? *ver : Semver();
}
void remove_installed_vendor(const std::string& vendor)
{
const boost::filesystem::path dir = boost::filesystem::path(data_dir()) / PRESET_SYSTEM_DIR;
boost::filesystem::remove(dir / (vendor + ".json"));
boost::filesystem::remove(dir / (vendor + ".opc"));
if (boost::filesystem::exists(dir / vendor))
boost::filesystem::remove_all(dir / vendor);
}
std::set<std::string> vendor_names_in(const boost::filesystem::path& dir)
{
std::set<std::string> names;
for (auto& dir_entry : boost::filesystem::directory_iterator(dir)) {
const auto& path = dir_entry.path();
if (Slic3r::is_json_file(path.string()) || path.extension() == ".opc")
names.insert(path.stem().string());
}
return names;
}
// A vendor's preset cache is the whole of its installation: it carries the presets,
// the vendor profile and the version they were built at, so where one ships nothing
// else needs copying. Unless the profile beside it claims a newer version — a cache
// generated before that profile was bumped is out of date, and a cache that cannot
// be read is no installation at all — and the vendor is installed the way it was
// before caches existed, as its profile and the preset JSONs it points at. Returns
// the version the cache is stamped with, invalid when it is not the form to install.
static Semver installable_cache_version(const boost::filesystem::path& dir, const std::string& vendor)
{
const auto cache_ver = Semver::parse(PresetBundle::peek_vendor_cache_version((dir / (vendor + ".opc")).string(), vendor));
if (! cache_ver)
return Semver::invalid();
const Semver profile_ver = get_version_from_json((dir / (vendor + ".json")).string());
return profile_ver.valid() && *cache_ver < profile_ver ? Semver::invalid() : *cache_ver;
}
Semver resource_vendor_version(const std::string& vendor)
{
const boost::filesystem::path dir = boost::filesystem::path(resources_dir()) / "profiles";
const Semver ver = installable_cache_version(dir, vendor);
return ver.valid() ? ver : get_version_from_json((dir / (vendor + ".json")).string());
}
bool install_vendor_bundles_from_resources(
const std::vector<std::string>& bundle_names,
const std::string& resource_subdir,
const std::string& data_subdir)
{
namespace fs = boost::filesystem;
fs::path rsrc_path = fs::path(Slic3r::resources_dir()) / resource_subdir;
fs::path vendor_path = fs::path(Slic3r::data_dir()) / data_subdir;
BOOST_LOG_TRIVIAL(info) << "Installing " << bundle_names.size() << " bundles from resources...";
for (const auto &bundle : bundle_names) {
try {
// Install the JSON file
auto path_in_rsrc = (rsrc_path / bundle).replace_extension(".json");
auto path_in_vendors = (vendor_path / bundle).replace_extension(".json");
auto cache_in_rsrc = (rsrc_path / bundle).replace_extension(".opc");
auto cache_in_vendors = (vendor_path / bundle).replace_extension(".opc");
// Either form of the vendor will do: a build may ship it as a cache alone.
if (!fs::exists(path_in_rsrc) && !fs::exists(cache_in_rsrc)) {
BOOST_LOG_TRIVIAL(warning) << "Bundle not found in resources: " << bundle;
return false;
}
// Create target directory if needed
if (!fs::exists(vendor_path))
fs::create_directories(vendor_path);
std::string error_message;
bool installed_cache = false;
if (installable_cache_version(rsrc_path, bundle).valid()) {
installed_cache = copy_file(cache_in_rsrc.string(), cache_in_vendors.string(), error_message, false) == CopyFileResult::SUCCESS;
if (! installed_cache)
BOOST_LOG_TRIVIAL(warning) << "Failed to copy " << bundle << ".opc: " << error_message;
} else {
boost::system::error_code ec;
fs::remove(cache_in_vendors, ec);
}
if (! installed_cache) {
CopyFileResult cfr = copy_file(path_in_rsrc.string(), path_in_vendors.string(), error_message, false);
if (cfr != CopyFileResult::SUCCESS) {
BOOST_LOG_TRIVIAL(error) << "Failed to copy " << bundle << ".json: " << error_message;
return false;
}
} else {
// Left in place, an earlier install's profile would shadow the cache.
boost::system::error_code ec;
fs::remove(path_in_vendors, ec);
}
// Copy the vendor directory (if it exists)
auto dir_in_rsrc = rsrc_path / bundle;
auto dir_in_vendors = vendor_path / bundle;
// Whatever is installed came from an earlier version of this vendor and
// would be parsed in place of the one being installed now.
if (fs::exists(dir_in_vendors))
fs::remove_all(dir_in_vendors);
if (! installed_cache && fs::exists(dir_in_rsrc) && fs::is_directory(dir_in_rsrc)) {
fs::create_directories(dir_in_vendors);
// Copy with file filter (same as PresetUpdater::install_bundles_rsrc)
// Filter out certain file types: .stl, .png, .svg, .jpeg, .jpg, .3mf
auto file_filter = [](const std::string name) -> bool {
return boost::iends_with(name, ".stl") ||
boost::iends_with(name, ".png") ||
boost::iends_with(name, ".svg") ||
boost::iends_with(name, ".jpeg") ||
boost::iends_with(name, ".jpg") ||
boost::iends_with(name, ".3mf");
};
copy_directory_recursively(dir_in_rsrc, dir_in_vendors, file_filter);
}
BOOST_LOG_TRIVIAL(info) << "Successfully installed bundle: " << bundle;
} catch (const std::exception& e) {
BOOST_LOG_TRIVIAL(error) << "Exception installing bundle " << bundle << ": " << e.what();
return false;
}
}
return true;
}
// m_printer_hold_alias survives reset() (and a cache body that failed partway
// in), so every full-bundle rebuild clears all five collections' maps by hand.
void PresetBundle::clear_printer_hold_aliases()
{
this->prints.m_printer_hold_alias.clear();
this->sla_prints.m_printer_hold_alias.clear();
this->filaments.m_printer_hold_alias.clear();
this->sla_materials.m_printer_hold_alias.clear();
this->printers.m_printer_hold_alias.clear();
}
//BBS: add json related logic, load system presets from json
std::pair<PresetsConfigSubstitutions, std::string> PresetBundle::load_system_presets_from_json(ForwardCompatibilitySubstitutionRule compatibility_rule)
{
@@ -2243,22 +2446,19 @@ std::pair<PresetsConfigSubstitutions, std::string> PresetBundle::load_system_pre
if (validation_mode)
dir = (boost::filesystem::path(data_dir())).make_preferred();
const auto load_t0 = std::chrono::steady_clock::now();
// The vendors below are loaded whole and against each other — the filament
// library first, then every other vendor with it as the base — so each parse
// is complete enough to be worth caching.
m_generate_vendor_caches = m_generate_vendor_caches || ! validation_mode;
PresetsConfigSubstitutions substitutions;
std::string errors_cummulative;
bool first = true;
std::vector<std::string> vendor_names;
// store all vendor names in vendor_names
for (auto& dir_entry : boost::filesystem::directory_iterator(dir)) {
std::string vendor_file = dir_entry.path().string();
if (!Slic3r::is_json_file(vendor_file))
continue;
std::string vendor_name = dir_entry.path().filename().string();
// Remove the .json suffix.
vendor_name.erase(vendor_name.size() - 5);
vendor_names.push_back(vendor_name);
}
bool first = true;
// Sorted, so any duplicate-preset warning below comes out in the same order on
// every run.
const std::set<std::string> vendor_names = vendor_names_in(dir);
// Separate ORCA_FILAMENT_LIBRARY from other vendors. It must be loaded
// first because other vendors' filaments may inherit from it via the
// `base_bundle` lookup in parse_subfile. The remaining vendors are
@@ -2274,8 +2474,13 @@ std::pair<PresetsConfigSubstitutions, std::string> PresetBundle::load_system_pre
}
// Step 1: Load ORCA_FILAMENT_LIBRARY into `this` synchronously.
if (!orca_lib_vendor.empty()) {
if (! orca_lib_vendor.empty()) {
try {
// Match a fresh launch before parsing: hold aliases and the error
// counter survive reset(), and would otherwise leak prior-cycle
// state into the library cache the load below writes.
this->clear_printer_hold_aliases();
this->m_errors = 0;
append(substitutions, this->load_vendor_configs_from_json(dir.string(), orca_lib_vendor, PresetBundle::LoadSystem, compatibility_rule).first);
first = false;
} catch (const std::runtime_error &err) {
@@ -2293,15 +2498,20 @@ std::pair<PresetsConfigSubstitutions, std::string> PresetBundle::load_system_pre
std::vector<PresetsConfigSubstitutions> parallel_substitutions(other_vendors.size());
std::vector<std::string> parallel_errors(other_vendors.size());
// The filament library version every vendor below is judged against. Fixed
// from here on — step 1 was the last thing that could touch the library on
// disk — so resolve it once instead of once per vendor.
const std::string lib_version = effective_lib_version(dir);
tbb::parallel_for(tbb::blocked_range<size_t>(0, other_vendors.size()),
[&](const tbb::blocked_range<size_t>& range) {
for (size_t i = range.begin(); i < range.end(); ++i) {
auto bundle = std::make_unique<PresetBundle>();
bundle->set_is_validation_mode(validation_mode);
bundle->set_generate_vendor_caches(m_generate_vendor_caches);
try {
auto result = bundle->load_vendor_configs_from_json(
dir.string(), other_vendors[i], PresetBundle::LoadSystem,
compatibility_rule, this);
dir.string(), other_vendors[i], PresetBundle::LoadSystem, compatibility_rule, this, lib_version);
parallel_substitutions[i] = std::move(result.first);
parallel_bundles[i] = std::move(bundle);
} catch (const std::runtime_error &err) {
@@ -2346,6 +2556,11 @@ std::pair<PresetsConfigSubstitutions, std::string> PresetBundle::load_system_pre
}
this->update_system_maps();
const auto load_ms = std::chrono::duration_cast<std::chrono::milliseconds>(
std::chrono::steady_clock::now() - load_t0).count();
BOOST_LOG_TRIVIAL(info) << "PresetBundle: " << vendor_names.size() << " vendor(s) loaded in " << load_ms << " ms";
//BBS: add config related logs
BOOST_LOG_TRIVIAL(debug) << __FUNCTION__ << boost::format(" finished, errors_cummulative %1%")%errors_cummulative;
return std::make_pair(std::move(substitutions), errors_cummulative);
@@ -4762,18 +4977,38 @@ void PresetBundle::load_config_file_config(const std::string &name_or_path, bool
//BBS: Load a config bundle file from json
std::pair<PresetsConfigSubstitutions, size_t> PresetBundle::load_vendor_configs_from_json(
const std::string &path, const std::string &vendor_name, LoadConfigBundleAttributes flags, ForwardCompatibilitySubstitutionRule compatibility_rule, const PresetBundle* base_bundle)
const std::string &dir, const std::string &vendor_name, LoadConfigBundleAttributes flags, ForwardCompatibilitySubstitutionRule compatibility_rule, const PresetBundle* base_bundle,
const std::string &lib_version_hint)
{
// Enable substitutions for user config bundle, throw an exception when loading a system profile.
ConfigSubstitutionContext substitution_context { compatibility_rule };
PresetsConfigSubstitutions substitutions;
//BBS: add config related logs
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << boost::format(" enter, path %1%, compatibility_rule %2%")%path.c_str()%compatibility_rule;
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << boost::format(" enter, path %1%, compatibility_rule %2%")%dir.c_str()%compatibility_rule;
if (flags.has(LoadConfigBundleAttribute::ResetUserProfile) || flags.has(LoadConfigBundleAttribute::LoadSystem))
// Reset this bundle, delete user profile files if SaveImported.
this->reset(flags.has(LoadConfigBundleAttribute::SaveImported));
// Orca: only a whole-vendor load has a cache — the vendor-only and filament-only
// scans want a slice of one. Validation reads the JSONs whatever is cached.
const boost::filesystem::path dir_path(dir);
const bool cacheable = flags.has(LoadConfigBundleAttribute::LoadSystem) && ! flags.has(LoadConfigBundleAttribute::LoadFilamentOnly);
if (cacheable && ! validation_mode && this->load_vendor_cache(dir_path, vendor_name, lib_version_hint)) {
size_t presets_loaded = 0;
for (const PresetCollection* coll : std::initializer_list<const PresetCollection*>{
&this->prints, &this->sla_prints, &this->filaments, &this->sla_materials, &this->printers })
presets_loaded += coll->m_presets.size() - coll->m_num_default_presets;
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << boost::format(", %1% served from its preset cache, %2% presets")%vendor_name%presets_loaded;
return std::make_pair(std::move(substitutions), presets_loaded);
}
// Orca: a build that ships preset caches installs them without the preset
// JSONs, so a vendor left to be parsed is parsed from the profiles in
// resources. An update does deliver JSONs into `dir`, and those win.
const std::string path = (validation_mode || boost::filesystem::exists(dir_path / (vendor_name + ".json")))
? dir : (boost::filesystem::path(resources_dir()) / "profiles").string();
// 1) load the vroot json and construct the vendor profile
VendorProfile vendor_profile(vendor_name);
std::string root_file = path + "/" + vendor_name + ".json";
@@ -5331,6 +5566,22 @@ std::pair<PresetsConfigSubstitutions, size_t> PresetBundle::load_vendor_configs_
}
}
// Orca: leave the vendor's cache in step with the profile just parsed, so the
// next run reads it instead. It is written where the vendor was looked for,
// even when the profile came from resources, and stamped with the version that
// profile claims — a profile without one cannot be judged for staleness later,
// and a cache nothing can invalidate is worse than none.
if (cacheable && m_generate_vendor_caches && vendor_profile.config_version.valid()) {
const std::string version = vendor_profile.config_version.to_string();
// The library is its own reference point; every other vendor's cache holds
// filaments resolved against it, and is stamped with the version in effect.
const std::string lib_version = vendor_name == ORCA_FILAMENT_LIBRARY ? version
: ! lib_version_hint.empty() ? lib_version_hint : effective_lib_version(dir_path);
if (! lib_version.empty() &&
! this->save_vendor_cache((dir_path / (vendor_name + ".opc")).string(), vendor_name, version, lib_version))
BOOST_LOG_TRIVIAL(warning) << "PresetBundle: failed to save vendor cache for " << vendor_name;
}
//BBS: add config related logs
BOOST_LOG_TRIVIAL(debug) << __FUNCTION__ << boost::format(", finished, presets_loaded %1%")%presets_loaded;
return std::make_pair(std::move(substitutions), presets_loaded);
@@ -5953,4 +6204,318 @@ bool BundleMetadata::save_to_json(const std::string& path) const
return false;
}
}
// ---- Preset cache file format (shared by the per-vendor cache) ----------
namespace {
#pragma pack(push, 1)
struct CacheFileHeader {
uint32_t magic;
uint32_t version;
uint64_t data_size;
uint32_t crc32;
};
#pragma pack(pop)
static_assert(sizeof(CacheFileHeader) == 20, "CacheFileHeader must be 20 bytes");
constexpr uint32_t CACHE_MAGIC = 0x4F52435A; // "ORCZ"
// Bump when the wire format changes in a way the schema fingerprint cannot
// detect: reordering, removing, or retyping a field of a hand-written
// serialize() (Preset, VendorProfile and its nested types), or when the
// cache's own layout or the meaning of its stamps changes (e.g. the move from
// one whole-bundle cache to one cache per vendor).
constexpr uint32_t CACHE_VERSION = 4;
// A cache stays usable as long as it was built from a vendor profile — and a
// filament library — at least as new as the ones now on disk. Profiles without
// a version cannot be judged this way and are never served from cache; where no
// profile sits beside the cache at all, nothing can be newer than it.
static bool cache_covers_version(const std::string& cached, const std::string& on_disk)
{
if (on_disk == PresetBundle::CACHE_ANY_VERSION)
return true;
const auto cached_ver = Semver::parse(cached);
const auto on_disk_ver = Semver::parse(on_disk);
return cached_ver && on_disk_ver && *cached_ver >= *on_disk_ver;
}
// Fingerprint of everything that determines the cache wire format: the app
// version and the DynamicPrintConfig option schema (key/type/ordinal/enum
// values — serialization_key_ordinal IS the config wire format). Any mismatch
// means bytes written by another build could deserialize into the wrong
// fields, so the cache is rejected wholesale before its body is read.
const std::string& compute_cache_schema_fingerprint()
{
// Constant for the lifetime of the process (print_config_def is immutable
// after static initialization), and asked for once per cache load and save.
static const std::string fingerprint = [] {
std::string schema;
schema += SLIC3R_VERSION;
schema += ';';
for (const auto& [key, def] : print_config_def.options) { // std::map => stable order
schema += key;
schema += '#'; schema += std::to_string(int(def.type));
schema += '@'; schema += std::to_string(def.serialization_key_ordinal);
for (const std::string& ev : def.enum_values) { schema += ','; schema += ev; }
schema += ';';
}
boost::crc_32_type crc;
crc.process_bytes(schema.data(), schema.size());
return std::to_string(crc.checksum());
}();
return fingerprint;
}
} // anonymous namespace
// static
bool PresetBundle::read_cache_blob(const std::string& path, std::string& out_blob)
{
try {
boost::nowide::ifstream ifs(path, std::ios::binary);
if (!ifs.is_open())
return false;
CacheFileHeader fhdr;
if (!ifs.read(reinterpret_cast<char*>(&fhdr), sizeof(fhdr)))
return false;
if (fhdr.magic != CACHE_MAGIC)
return false;
if (fhdr.data_size == 0 || fhdr.data_size > 512u * 1024u * 1024u)
return false;
out_blob.assign(fhdr.data_size, '\0');
if (!ifs.read(&out_blob[0], static_cast<std::streamsize>(fhdr.data_size)))
return false;
boost::crc_32_type crc;
crc.process_bytes(out_blob.data(), out_blob.size());
if (crc.checksum() != fhdr.crc32) {
BOOST_LOG_TRIVIAL(warning) << "SystemPresetsCache: CRC mismatch: " << path;
return false;
}
return true;
} catch (const std::exception& e) {
BOOST_LOG_TRIVIAL(warning) << "SystemPresetsCache: read failed (" << path << "): " << e.what();
return false;
}
}
// static
bool PresetBundle::write_cache_blob(const std::string& path, const std::string& blob)
{
boost::crc_32_type crc;
crc.process_bytes(blob.data(), blob.size());
try {
boost::filesystem::create_directories(boost::filesystem::path(path).parent_path());
boost::nowide::ofstream ofs(path, std::ios::binary | std::ios::trunc);
if (!ofs.is_open()) {
BOOST_LOG_TRIVIAL(warning) << "SystemPresetsCache: cannot open for writing: " << path;
return false;
}
CacheFileHeader fhdr;
fhdr.magic = CACHE_MAGIC;
fhdr.version = CACHE_VERSION;
fhdr.data_size = static_cast<uint64_t>(blob.size());
fhdr.crc32 = crc.checksum();
ofs.write(reinterpret_cast<const char*>(&fhdr), sizeof(fhdr));
ofs.write(blob.data(), static_cast<std::streamsize>(blob.size()));
ofs.close(); // flush; close() raises failbit on error
if (! ofs.good())
BOOST_LOG_TRIVIAL(warning) << "SystemPresetsCache: write failed (" << path << ")";
return ofs.good();
} catch (const std::exception& e) {
BOOST_LOG_TRIVIAL(warning) << "SystemPresetsCache: write failed (" << path << "): " << e.what();
return false;
}
}
// ---- Per-vendor preset cache ---------------------------------------------
// Serializes every non-default, non-external preset of one collection plus
// its m_printer_hold_alias. Defaults are constructor-derived state and are
// skipped; load re-installs the loaded presets after the constructor's
// defaults, exactly like the JSON parse path (reset-to-defaults + sorted
// append).
void PresetBundle::save_collection(cereal::BinaryOutputArchive& ar, const PresetCollection& coll)
{
uint64_t count = 0;
for (const Preset& p : coll.m_presets)
if (! p.is_default && ! p.is_external)
++ count;
ar(count);
for (const Preset& p : coll.m_presets) {
if (p.is_default || p.is_external)
continue;
std::string vendor_id = p.vendor ? p.vendor->id : std::string();
ar(vendor_id, p);
}
// unordered containers -> sorted, for byte-deterministic output (parity test)
std::map<std::string, std::set<std::string>> hold;
for (const auto& entry : coll.m_printer_hold_alias)
hold.emplace(entry.first, std::set<std::string>(entry.second.begin(), entry.second.end()));
ar(hold);
}
void PresetBundle::load_collection(cereal::BinaryInputArchive& ar, PresetCollection& coll, const VendorMap& vendors)
{
coll.m_printer_hold_alias.clear();
// Drop everything but the constructor-installed defaults, like PresetCollection::reset().
coll.m_presets.erase(coll.m_presets.begin() + coll.m_num_default_presets, coll.m_presets.end());
// Mirror PresetCollection::reset(), which follows the same truncation with
// select_preset(0): re-point selection/edited-preset state at the default
// preset before the loaded presets are appended below, so a cache hit onto
// an already-populated bundle (e.g. a reload) leaves selection state
// identical to the JSON-parse path instead of carrying over a stale index.
coll.select_preset(0);
uint64_t count = 0;
ar(count);
for (uint64_t i = 0; i < count; ++ i) {
std::string vendor_id;
Preset preset(coll.m_type, std::string());
ar(vendor_id, preset);
if (! vendor_id.empty()) {
auto it = vendors.find(vendor_id);
if (it == vendors.end())
throw std::runtime_error("vendor cache references unknown vendor: " + vendor_id);
preset.vendor = &it->second;
}
coll.m_presets.emplace_back(std::move(preset));
}
std::map<std::string, std::set<std::string>> hold;
ar(hold);
for (auto& entry : hold)
coll.m_printer_hold_alias.emplace(entry.first, std::unordered_set<std::string>(entry.second.begin(), entry.second.end()));
}
// ---- Per-vendor preset cache implementation ------------------------------
bool PresetBundle::save_vendor_cache(const std::string& cache_path, const std::string& vendor_name,
const std::string& vendor_version, const std::string& lib_version) const
{
try {
std::ostringstream body(std::ios::binary);
{
cereal::BinaryOutputArchive ar(body);
ar(CACHE_VERSION);
ar(compute_cache_schema_fingerprint());
ar(vendor_name, vendor_version, lib_version);
ar(this->vendors);
save_collection(ar, this->prints);
save_collection(ar, this->sla_prints);
save_collection(ar, this->filaments);
save_collection(ar, this->sla_materials);
save_collection(ar, this->printers);
ar(this->m_config_maps, this->m_filament_id_maps);
ar(this->obsolete_presets.prints, this->obsolete_presets.sla_prints,
this->obsolete_presets.filaments, this->obsolete_presets.sla_materials,
this->obsolete_presets.printers);
ar(this->m_errors);
}
return write_cache_blob(cache_path, body.str());
} catch (const std::exception& e) {
BOOST_LOG_TRIVIAL(warning) << "PresetBundle: failed to save vendor cache " << cache_path << ": " << e.what();
return false;
}
}
// static
std::string PresetBundle::peek_vendor_cache_version(const std::string& cache_path, const std::string& expected_vendor_name)
{
try {
boost::nowide::ifstream ifs(cache_path, std::ios::binary);
CacheFileHeader fhdr;
if (! ifs.read(reinterpret_cast<char*>(&fhdr), sizeof(fhdr)) || fhdr.magic != CACHE_MAGIC)
return {};
// Only the head of the body is read, and its CRC left unverified: the stamps
// sit at the front, this answers "what version is this vendor at?" once per
// vendor on every update check, and reading tens of megabytes to do so is not
// worth it. A stamp that comes out garbled fails to parse as a version, which
// is the same answer as none.
std::string head(static_cast<size_t>(std::min<uint64_t>(fhdr.data_size, 1024)), '\0');
if (! ifs.read(&head[0], static_cast<std::streamsize>(head.size())))
return {};
std::istringstream body(head, std::ios::binary);
cereal::BinaryInputArchive ar(body);
uint32_t cache_version = 0;
ar(cache_version);
std::string fingerprint, vendor_name, vendor_version;
ar(fingerprint);
ar(vendor_name, vendor_version);
// The fingerprint is deliberately not checked: the version a cache carries
// is what this build installed, whether or not this build can still read it.
if (cache_version != CACHE_VERSION || vendor_name != expected_vendor_name)
return {};
return vendor_version;
} catch (const std::exception&) {
return {};
}
}
bool PresetBundle::load_vendor_cache(const boost::filesystem::path& dir, const std::string& vendor_name, const std::string& lib_version_hint)
{
// Whichever cache answers is judged against the vendor as installed in `dir`:
// the profile there, or — with none, as when the cache is the whole of the
// installation — nothing, since nothing on disk can then be newer than it.
// Plus the filament library in effect, which a vendor's filaments were
// resolved against when its cache was built.
const boost::filesystem::path profile = dir / (vendor_name + ".json");
const std::string version = boost::filesystem::exists(profile) ? get_vendor_cache_version(profile.string())
: std::string(CACHE_ANY_VERSION);
const std::string lib_version = ! lib_version_hint.empty() ? lib_version_hint : effective_lib_version(dir);
const boost::filesystem::path rsrc = boost::filesystem::path(resources_dir()) / "profiles";
return this->load_vendor_cache((dir / (vendor_name + ".opc")).string(), vendor_name, version, lib_version)
|| (dir != rsrc && this->load_vendor_cache((rsrc / (vendor_name + ".opc")).string(), vendor_name, version, lib_version));
}
bool PresetBundle::load_vendor_cache(const std::string& cache_path, const std::string& expected_vendor_name,
const std::string& expected_vendor_version, const std::string& expected_lib_version)
{
std::string blob;
if (! read_cache_blob(cache_path, blob))
return false;
try {
// Read in place: an istringstream would copy the blob (tens of MB for
// the largest vendors) once more just to stream over it.
boost::iostreams::stream<boost::iostreams::array_source> body(blob.data(), blob.size());
cereal::BinaryInputArchive ar(body);
uint32_t cache_version = 0;
ar(cache_version);
if (cache_version != CACHE_VERSION)
return false;
std::string fingerprint;
ar(fingerprint);
if (fingerprint != compute_cache_schema_fingerprint())
return false;
std::string vendor_name, vendor_version, lib_version;
ar(vendor_name, vendor_version, lib_version);
if (vendor_name != expected_vendor_name ||
! cache_covers_version(vendor_version, expected_vendor_version) ||
! cache_covers_version(lib_version, expected_lib_version))
return false;
ar(this->vendors);
load_collection(ar, this->prints, this->vendors);
load_collection(ar, this->sla_prints, this->vendors);
load_collection(ar, this->filaments, this->vendors);
load_collection(ar, this->sla_materials, this->vendors);
load_collection(ar, this->printers, this->vendors);
ar(this->m_config_maps, this->m_filament_id_maps);
ar(this->obsolete_presets.prints, this->obsolete_presets.sla_prints,
this->obsolete_presets.filaments, this->obsolete_presets.sla_materials,
this->obsolete_presets.printers);
ar(this->m_errors);
return true;
} catch (const std::exception& e) {
BOOST_LOG_TRIVIAL(warning) << "PresetBundle: rejecting vendor cache " << cache_path << ": " << e.what();
// Restore a clean state so the caller can fall back to the JSON parse.
this->reset(false);
this->vendors.clear();
this->m_config_maps.clear();
this->m_filament_id_maps.clear();
this->m_errors = 0;
// A mid-body failure may have left collections the deserialization never
// reached (each load_collection clears its own collection's map only when
// it runs) with stale aliases.
this->clear_printer_hold_aliases();
return false;
}
}
} // namespace Slic3r

View File

@@ -6,6 +6,7 @@
#include "enum_bitmask.hpp"
#include <memory>
#include <set>
#include <shared_mutex>
#include <unordered_map>
#include <optional>
@@ -13,6 +14,11 @@
#include <boost/filesystem/path.hpp>
#include <unordered_set>
namespace cereal {
class BinaryInputArchive;
class BinaryOutputArchive;
}
#define DEFAULT_USER_FOLDER_NAME "default"
#define BUNDLE_STRUCTURE_JSON_NAME "bundle_structure.json"
@@ -170,6 +176,44 @@ struct PresetBundleMetadata
class PresetBundle
{
public:
// ---- Per-vendor preset cache --------------------------------------------
// One cache file per vendor (plus the Orca filament library), stamped with
// the vendor's own profile version rather than a directory scan.
// The cache is not something a caller loads from: a vendor is loaded with
// load_vendor_configs_from_json, which comes from the cache whenever one covers
// it. What is public here is what the cache's own tests drive directly.
// Save this bundle's slice belonging to one vendor (vendor_name at
// vendor_version), built against the given filament library version.
bool save_vendor_cache(const std::string& cache_path, const std::string& vendor_name,
const std::string& vendor_version, const std::string& lib_version) const;
// Expected version meaning "no profile sits beside this cache", so nothing can
// be newer than it and only its own integrity decides whether it is used. Not
// the same as an empty version, which means a profile is there but unreadable.
static constexpr const char* CACHE_ANY_VERSION = "*";
// Load a validated per-vendor cache into this bundle. Rejects (returns
// false) unless the cache version, schema fingerprint and vendor name match
// and the cache was built from a vendor profile and filament library at
// least as new as the expected ones. Profiles without a version (empty
// string) are never served from cache.
bool load_vendor_cache(const std::string& cache_path, const std::string& expected_vendor_name,
const std::string& expected_vendor_version, const std::string& expected_lib_version);
// Read the profile version a cache was stamped with, without deserializing its
// presets. Empty if the file is unreadable, not a cache this build understands,
// or not this vendor's. This is how an installed vendor's version is known when
// only its cache is installed.
static std::string peek_vendor_cache_version(const std::string& cache_path, const std::string& expected_vendor_name);
// Enable writing a per-vendor cache after a JSON parse (off by default). Only
// for bundles whose parses are complete — load_system_presets_from_json loads
// the filament library first and every other vendor against it, so its parses
// qualify; a wizard's one-off parse of a single vendor does not.
void set_generate_vendor_caches(bool enable) { m_generate_vendor_caches = enable; }
static DynamicPrintConfig construct_full_config(Preset &in_printer_preset,
Preset &in_print_preset,
const DynamicPrintConfig &project_config,
@@ -444,8 +488,15 @@ public:
/*std::pair<PresetsConfigSubstitutions, size_t> load_configbundle(
const std::string &path, LoadConfigBundleAttributes flags, ForwardCompatibilitySubstitutionRule compatibility_rule);*/
//Orca: load config bundle from json, pass the base bundle to support cross vendor inheritance
// Orca: `dir` is where the vendor is looked for — its own directory, whether or
// not the profile JSONs are still there. A whole-vendor load comes from the
// vendor's preset cache whenever one covers the profile on disk, and is parsed
// from the JSONs (falling back to the ones in resources) only when none does.
// `lib_version_hint` is the filament library version in effect, when the caller
// loads many vendors and has already resolved it once; empty resolves it here.
std::pair<PresetsConfigSubstitutions, size_t> load_vendor_configs_from_json(
const std::string &path, const std::string &vendor_name, LoadConfigBundleAttributes flags, ForwardCompatibilitySubstitutionRule compatibility_rule, const PresetBundle* base_bundle = nullptr);
const std::string &dir, const std::string &vendor_name, LoadConfigBundleAttributes flags, ForwardCompatibilitySubstitutionRule compatibility_rule, const PresetBundle* base_bundle = nullptr,
const std::string &lib_version_hint = {});
// Export a config bundle file containing all the presets and the names of the active presets.
//void export_configbundle(const std::string &path, bool export_system_settings = false, bool export_physical_printers = false);
@@ -521,7 +572,38 @@ public:
// compatible_prints references a deleted (unknown) or renamed (old) preset name.
bool check_preset_references() const;
// Merge one vendor's presets with the other vendor's presets, report duplicates.
// Public so per-vendor-cache consumers (e.g. the setup wizard) can assemble a
// bundle out of several per-vendor caches loaded into separate PresetBundle instances.
std::vector<std::string> merge_presets(PresetBundle &&other);
private:
// Load one vendor from its preset cache: the one in `dir`, or — when that is
// missing or stale — the one shipped in resources/profiles, both judged against
// the vendor as installed in `dir` and against the filament library in effect
// (resolved here unless the caller passes the already-resolved version).
// False, with this bundle left clean, when neither is usable and the vendor has
// to be parsed. This is how load_vendor_configs_from_json reads a cache.
bool load_vendor_cache(const boost::filesystem::path& dir, const std::string& vendor_name, const std::string& lib_version_hint = {});
// Read raw cache blob: verify magic, size, CRC.
static bool read_cache_blob(const std::string& path, std::string& out_blob);
// Write a cache blob with the standard 20-byte file header. False when the
// file could not be opened or written whole.
static bool write_cache_blob(const std::string& path, const std::string& blob);
// (De)serialization of one collection's slice for the per-vendor cache:
// every non-default, non-external preset (system or user) plus
// m_printer_hold_alias. See save_vendor_cache/load_vendor_cache.
static void save_collection(cereal::BinaryOutputArchive& ar, const PresetCollection& coll);
static void load_collection(cereal::BinaryInputArchive& ar, PresetCollection& coll, const VendorMap& vendors);
// Clear every collection's m_printer_hold_alias, which reset() leaves alone.
void clear_printer_hold_aliases();
// Whether to (re)write a per-vendor cache after a JSON parse.
bool m_generate_vendor_caches { false };
// Orca: validation only - flag any printer with two or more compatible
// filament presets sharing one filament_id (ambiguous AMS subtype match).
bool check_duplicate_filament_subtypes() const;
@@ -529,8 +611,6 @@ private:
//std::pair<PresetsConfigSubstitutions, std::string> load_system_presets(ForwardCompatibilitySubstitutionRule compatibility_rule);
//BBS: add json related logic
std::pair<PresetsConfigSubstitutions, std::string> load_system_presets_from_json(ForwardCompatibilitySubstitutionRule compatibility_rule);
// Merge one vendor's presets with the other vendor's presets, report duplicates.
std::vector<std::string> merge_presets(PresetBundle &&other);
// Update the multicolor information for filaments.
void update_filament_multi_color();
// Update renamed_from and alias maps of system profiles.
@@ -565,6 +645,38 @@ private:
ENABLE_ENUM_BITMASK_OPERATORS(PresetBundle::LoadConfigBundleAttribute)
// True if `vendor` is installed in data_dir()/system. A build that ships preset
// caches installs the cache alone, so it — not the profile — marks a vendor
// installed, and either one on its own counts.
extern bool is_vendor_installed(const std::string& vendor);
// The version of the installed vendor: what its profile claims, or what its cache
// was stamped with where only the cache is installed. Invalid Semver if neither is.
extern Semver installed_vendor_version(const std::string& vendor);
// Remove every form `vendor` can be installed as from data_dir()/system: its
// profile, its preset cache, and its preset directory.
extern void remove_installed_vendor(const std::string& vendor);
// The vendors `dir` holds, sorted: one is named by its profile or, in a build that
// ships preset caches instead of the raw profile JSONs, by its cache alone.
extern std::set<std::string> vendor_names_in(const boost::filesystem::path& dir);
// The version a build ships `vendor` at: whichever of its preset cache and its
// profile is newer, that being the one installing lays down. Invalid Semver if the
// build ships neither.
extern Semver resource_vendor_version(const std::string& vendor);
// Install vendors from the resources directory into the data directory, each as
// its preset cache or as its profile and preset JSONs — whichever of the two the
// build ships at the newer version. Anything the previous install of that vendor
// left behind goes, so only the form just installed is there to be loaded.
// bundle_names: vendor names, without extension.
// Returns false on the first vendor that cannot be installed.
extern bool install_vendor_bundles_from_resources(const std::vector<std::string>& bundle_names,
const std::string& resource_subdir = "profiles",
const std::string& data_subdir = "system");
} // namespace Slic3r
#endif /* slic3r_PresetBundle_hpp_ */

View File

@@ -2410,7 +2410,8 @@ namespace cereal {
archive(serialization_key_ordinal);
assert(serialization_key_ordinal > 0);
auto it = Slic3r::print_config_def.by_serialization_key_ordinal.find(serialization_key_ordinal);
assert(it != Slic3r::print_config_def.by_serialization_key_ordinal.end());
if (it == Slic3r::print_config_def.by_serialization_key_ordinal.end())
throw std::runtime_error("VendorCache: unknown serialization_key_ordinal " + std::to_string(serialization_key_ordinal) + " - cache is stale");
config.set_key_value(it->second->opt_key, it->second->load_option_from_archive(archive));
}
}

View File

@@ -190,6 +190,19 @@ public:
os << self.to_string();
return os;
}
// cereal: round-trip through the standard 3-part string (major.minor.patch).
// to_string() uses a BBS 4-part format that semver_parse() cannot read back.
template<class Archive>
std::string save_minimal(const Archive&) const { return to_string_sf(); }
template<class Archive>
void load_minimal(const Archive&, const std::string& s) {
auto v = Semver::parse(s);
if (! v)
throw std::runtime_error("Semver: cannot parse serialized version: " + s);
*this = std::move(*v);
}
private:
semver_t ver;

View File

@@ -722,15 +722,6 @@ void copy_directory_recursively(const boost::filesystem::path& source,
std::function<bool(const std::string)> filter = nullptr,
bool merge_mode = false);
// Install vendor bundles from resources directory to data directory
// bundle_names: vector of vendor bundle names (without .json extension)
// resource_subdir: subdirectory under resources_dir() (default: "profiles")
// data_subdir: subdirectory under data_dir() (default: "system")
// Returns: true if all bundles installed successfully, false otherwise
bool install_vendor_bundles_from_resources(const std::vector<std::string>& bundle_names,
const std::string& resource_subdir = "profiles",
const std::string& data_subdir = "system");
// Orca: Since 1.7.9 Boost deprecated save_string_file and load_string_file, copy and modified from boost 1.7.8
void save_string_file(const boost::filesystem::path& p, const std::string& str);
void load_string_file(const boost::filesystem::path& p, std::string& str);

View File

@@ -1724,76 +1724,6 @@ void copy_directory_recursively(const boost::filesystem::path& source,
return;
}
bool install_vendor_bundles_from_resources(
const std::vector<std::string>& bundle_names,
const std::string& resource_subdir,
const std::string& data_subdir)
{
namespace fs = boost::filesystem;
fs::path rsrc_path = fs::path(Slic3r::resources_dir()) / resource_subdir;
fs::path vendor_path = fs::path(Slic3r::data_dir()) / data_subdir;
BOOST_LOG_TRIVIAL(info) << "Installing " << bundle_names.size() << " bundles from resources...";
for (const auto &bundle : bundle_names) {
try {
// Install the JSON file
auto path_in_rsrc = (rsrc_path / bundle).replace_extension(".json");
auto path_in_vendors = (vendor_path / bundle).replace_extension(".json");
if (!fs::exists(path_in_rsrc)) {
BOOST_LOG_TRIVIAL(warning) << "Bundle not found in resources: " << bundle;
return false;
}
// Create target directory if needed
if (!fs::exists(vendor_path))
fs::create_directories(vendor_path);
// Copy JSON file
std::string error_message;
CopyFileResult cfr = copy_file(path_in_rsrc.string(), path_in_vendors.string(), error_message, false);
if (cfr != CopyFileResult::SUCCESS) {
BOOST_LOG_TRIVIAL(error) << "Failed to copy " << bundle << ".json: " << error_message;
return false;
}
// Copy the vendor directory (if it exists)
auto dir_in_rsrc = rsrc_path / bundle;
auto dir_in_vendors = vendor_path / bundle;
if (fs::exists(dir_in_rsrc) && fs::is_directory(dir_in_rsrc)) {
// Remove existing directory
if (fs::exists(dir_in_vendors))
fs::remove_all(dir_in_vendors);
fs::create_directories(dir_in_vendors);
// Copy with file filter (same as PresetUpdater::install_bundles_rsrc)
// Filter out certain file types: .stl, .png, .svg, .jpeg, .jpg, .3mf
auto file_filter = [](const std::string name) -> bool {
return boost::iends_with(name, ".stl") ||
boost::iends_with(name, ".png") ||
boost::iends_with(name, ".svg") ||
boost::iends_with(name, ".jpeg") ||
boost::iends_with(name, ".jpg") ||
boost::iends_with(name, ".3mf");
};
copy_directory_recursively(dir_in_rsrc, dir_in_vendors, file_filter);
}
BOOST_LOG_TRIVIAL(info) << "Successfully installed bundle: " << bundle;
} catch (const std::exception& e) {
BOOST_LOG_TRIVIAL(error) << "Exception installing bundle " << bundle << ": " << e.what();
return false;
}
}
return true;
}
void save_string_file(const boost::filesystem::path& p, const std::string& str)
{
boost::nowide::ofstream file;

View File

@@ -432,14 +432,9 @@ const Snapshot& SnapshotDB::take_snapshot(const AppConfig &app_config, Snapshot:
cfg.models_variants_installed.erase(it ++);
else
++ it;
// Read the active config bundle, parse the config version.
PresetBundle bundle;
//BBS: change directoties by design
//bundle.load_configbundle((data_dir / PRESET_SYSTEM_DIR / (cfg.name + ".ini")).string(), PresetBundle::LoadConfigBundleAttribute::LoadVendorOnly, ForwardCompatibilitySubstitutionRule::EnableSilent);
bundle.load_vendor_configs_from_json((data_dir/PRESET_SYSTEM_DIR).string(), cfg.name, PresetBundle::LoadConfigBundleAttribute::LoadVendorOnly, ForwardCompatibilitySubstitutionRule::EnableSilent);
for (const auto &vp : bundle.vendors)
if (vp.second.id == cfg.name)
cfg.version.config_version = vp.second.config_version;
// Orca: the version the vendor is installed at, read from its profile or —
// where the cache is the whole installation — from the cache's own stamp.
cfg.version.config_version = installed_vendor_version(cfg.name);
snapshot.vendor_configs.emplace_back(std::move(cfg));
}

View File

@@ -66,41 +66,41 @@ using Config::SnapshotDB;
// Configuration data structures extensions needed for the wizard
//BBS: set BBL as default
bool Bundle::load(fs::path source_path, bool ais_in_resources, bool ais_bbl_bundle)
bool Bundle::load(fs::path dir, const std::string &vendor_name, bool ais_in_resources, bool ais_bbl_bundle)
{
this->preset_bundle = std::make_unique<PresetBundle>();
this->is_in_resources = ais_in_resources;
this->is_bbl_bundle = ais_bbl_bundle;
std::string path_string = source_path.string();
std::string parent_path = source_path.parent_path().string();
//BBS: add json logic for vendor bundles
std::string vendor_name = source_path.filename().string();
if (Slic3r::is_json_file(path_string)) {
// Remove the .json suffix.
vendor_name.erase(vendor_name.size() - 5);
}
else
// Orca: served from the vendor's preset cache where one covers it — which is
// how a shipped build carries its vendors — and parsed from the JSONs otherwise.
// A vendor that can be neither read nor parsed — a cache the build cannot use
// with the preset JSONs behind it pruned, say — is one the wizard cannot offer.
// Every other vendor still can be, so it is left out rather than thrown over.
size_t presets_loaded = 0;
try {
auto [config_substitutions, loaded] = preset_bundle->load_vendor_configs_from_json(
dir.string(), vendor_name, PresetBundle::LoadConfigBundleAttribute::LoadSystem, ForwardCompatibilitySubstitutionRule::Disable);
UNUSED(config_substitutions);
// No substitutions shall be reported when loading a system config bundle, no substitutions are allowed.
assert(config_substitutions.empty());
presets_loaded = loaded;
} catch (const std::exception &e) {
BOOST_LOG_TRIVIAL(fatal) << boost::format("Vendor bundle: `%1%`: cannot be loaded, leaving it out: %2%") % vendor_name % e.what();
return false;
// Throw when parsing invalid configuration. Only valid configuration is supposed to be provided over the air.
//BBS: add json logic for vendor bundles
auto [config_substitutions, presets_loaded] = preset_bundle->load_vendor_configs_from_json(
parent_path, vendor_name, PresetBundle::LoadConfigBundleAttribute::LoadSystem, ForwardCompatibilitySubstitutionRule::Disable);
UNUSED(config_substitutions);
// No substitutions shall be reported when loading a system config bundle, no substitutions are allowed.
assert(config_substitutions.empty());
}
auto first_vendor = preset_bundle->vendors.begin();
if (first_vendor == preset_bundle->vendors.end()) {
BOOST_LOG_TRIVIAL(error) << boost::format("Vendor bundle: `%1%`: No vendor information defined, cannot install.") % path_string;
BOOST_LOG_TRIVIAL(error) << boost::format("Vendor bundle: `%1%`: No vendor information defined, cannot install.") % vendor_name;
return false;
}
if (presets_loaded == 0) {
BOOST_LOG_TRIVIAL(error) << boost::format("Vendor bundle: `%1%`: No profile loaded.") % path_string;
BOOST_LOG_TRIVIAL(error) << boost::format("Vendor bundle: `%1%`: No profile loaded.") % vendor_name;
return false;
}
}
BOOST_LOG_TRIVIAL(trace) << boost::format("Vendor bundle: `%1%`: %2% profiles loaded.") % path_string % presets_loaded;
BOOST_LOG_TRIVIAL(trace) << boost::format("Vendor bundle: `%1%`: %2% profiles loaded.") % vendor_name % presets_loaded;
this->vendor_profile = &first_vendor->second;
return true;
}
@@ -125,15 +125,10 @@ BundleMap BundleMap::load()
//Orca: add custom as default
//Orca: add json logic for vendor bundle
auto orca_bundle_path = (vendor_dir / PresetBundle::ORCA_DEFAULT_BUNDLE).replace_extension(".json");
auto orca_bundle_rsrc = false;
if (!boost::filesystem::exists(orca_bundle_path)) {
orca_bundle_path = (rsrc_vendor_dir / PresetBundle::ORCA_DEFAULT_BUNDLE).replace_extension(".json");
orca_bundle_rsrc = true;
}
{
const bool from_rsrc = ! is_vendor_installed(PresetBundle::ORCA_DEFAULT_BUNDLE);
Bundle bbl_bundle;
if (bbl_bundle.load(std::move(orca_bundle_path), orca_bundle_rsrc, true))
if (bbl_bundle.load(from_rsrc ? rsrc_vendor_dir : vendor_dir, PresetBundle::ORCA_DEFAULT_BUNDLE, from_rsrc, true))
res.emplace(PresetBundle::ORCA_DEFAULT_BUNDLE, std::move(bbl_bundle));
}
@@ -141,18 +136,13 @@ BundleMap BundleMap::load()
// and then additionally from resources/profiles.
bool is_in_resources = false;
for (auto dir : { &vendor_dir, &rsrc_vendor_dir }) {
for (const auto &dir_entry : boost::filesystem::directory_iterator(*dir)) {
//BBS: add json logic for vendor bundle
if (Slic3r::is_json_file(dir_entry.path().string())) {
std::string id = dir_entry.path().stem().string(); // stem() = filename() without the trailing ".json" part
for (const std::string &id : vendor_names_in(*dir)) {
// Don't load this bundle if we've already loaded it.
if (res.find(id) != res.end()) { continue; }
// Don't load this bundle if we've already loaded it.
if (res.find(id) != res.end()) { continue; }
Bundle bundle;
if (bundle.load(dir_entry.path(), is_in_resources))
res.emplace(std::move(id), std::move(bundle));
}
Bundle bundle;
if (bundle.load(*dir, id, is_in_resources))
res.emplace(id, std::move(bundle));
}
is_in_resources = true;

View File

@@ -71,9 +71,11 @@ struct Bundle
Bundle() = default;
Bundle(Bundle&& other);
// Load the vendor `vendor_name` as it is installed in `dir`, from its preset
// cache or its profile JSONs, whichever is usable.
// Returns false if not loaded. Reason for that is logged as boost::log error.
//BBS: set BBL as default
bool load(fs::path source_path, bool is_in_resources, bool is_bbl_bundle = false);
bool load(fs::path dir, const std::string &vendor_name, bool is_in_resources, bool is_bbl_bundle = false);
const std::string& vendor_id() const { return vendor_profile->id; }
};

View File

@@ -2201,25 +2201,14 @@ bool CreatePrinterPresetDialog::load_system_and_user_presets_with_curr_model(Pre
} else {
selected_vendor_id = m_printer_preset_vendor_selected.id;
if (boost::filesystem::exists(boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR / selected_vendor_id)) {
preset_path = (boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR).string();
} else if (boost::filesystem::exists(boost::filesystem::path(Slic3r::resources_dir()) / "profiles" / selected_vendor_id)) {
preset_path = (boost::filesystem::path(Slic3r::resources_dir()) / "profiles").string();
}
if (preset_path.empty()) {
BOOST_LOG_TRIVIAL(info) << "Preset path was not found";
MessageDialog dlg(this, _L("Preset path was not found; please reselect vendor."), wxString(SLIC3R_APP_FULL_NAME) + " - " + _L("Info"),
wxYES_NO | wxYES_DEFAULT | wxCENTRE);
dlg.ShowModal();
return false;
}
try {
// Pass the app's preset bundle (which already holds OrcaFilamentLibrary) as the base
// bundle so vendor filaments that inherit OFL bases resolve via the existing
// cross-vendor inheritance path.
temp_preset_bundle.load_vendor_configs_from_json(preset_path, selected_vendor_id,
// Orca: served from the vendor's preset cache where one covers it — a shipped
// build carries that instead of the raw preset JSONs — and parsed otherwise.
temp_preset_bundle.load_vendor_configs_from_json((boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR).string(),
selected_vendor_id,
PresetBundle::LoadConfigBundleAttribute::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent,
wxGetApp().preset_bundle);

View File

@@ -1,7 +1,9 @@
#include "WebGuideDialog.hpp"
#include "ConfigWizard.hpp"
#include <boost/algorithm/string/join.hpp>
#include <boost/filesystem/operations.hpp>
#include <boost/nowide/fstream.hpp>
#include <boost/filesystem/path.hpp>
#include <boost/iostreams/detail/select.hpp>
#include <boost/log/trivial.hpp>
@@ -9,6 +11,7 @@
#include "I18N.hpp"
#include "libslic3r/AppConfig.hpp"
#include "libslic3r/Config.hpp"
#include "libslic3r/Preset.hpp"
#include "libslic3r/PresetBundle.hpp"
#include "slic3r/GUI/wxExtensions.hpp"
#include "slic3r/GUI/GUI_App.hpp"
@@ -41,8 +44,6 @@ using namespace nlohmann;
namespace Slic3r { namespace GUI {
json m_ProfileJson;
static wxString update_custom_filaments()
{
json m_Res = json::object();
@@ -190,12 +191,10 @@ GuideFrame::GuideFrame(GUI_App *pGUI, long style)
GuideFrame::~GuideFrame()
{
m_destroy = true;
if (m_load_task && m_load_task->joinable()) {
*m_cancel_token = true; // stop the loading thread and any queued CallAfter lambdas before join
if (m_load_task && m_load_task->joinable())
m_load_task->join();
delete m_load_task;
m_load_task = nullptr;
}
m_load_task.reset();
if (m_browser) {
delete m_browser;
m_browser = nullptr;
@@ -301,15 +300,71 @@ void GuideFrame::OnNavigationRequest(wxWebViewEvent &evt)
/**
* Callback invoked when a navigation request was accepted
*/
// The empty shape every profile-loading path starts from or falls back to.
void GuideFrame::reset_profile_json()
{
m_ProfileJson["model"] = json::array();
m_ProfileJson["machine"] = json::object();
m_ProfileJson["filament"] = json::object();
m_ProfileJson["process"] = json::array();
}
void GuideFrame::init_guide_paths()
{
m_ProfileJson = json::parse("{}");
reset_profile_json();
vendor_dir = (boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR).make_preferred();
rsrc_vendor_dir = (boost::filesystem::path(resources_dir()) / "profiles").make_preferred();
orca_bundle_rsrc = true;
if (boost::filesystem::exists(vendor_dir)) {
for (const auto& entry : boost::filesystem::directory_iterator(vendor_dir)) {
if (!boost::filesystem::is_directory(entry) &&
boost::iequals(entry.path().extension().string(), ".json") &&
!boost::iequals(entry.path().stem().string(), PresetBundle::ORCA_FILAMENT_LIBRARY)) {
orca_bundle_rsrc = false;
break;
}
}
}
auto lib_json = boost::filesystem::path(PresetBundle::ORCA_FILAMENT_LIBRARY).replace_extension(".json");
m_OrcaFilaLibPath = boost::filesystem::exists(vendor_dir / lib_json)
? (vendor_dir / PresetBundle::ORCA_FILAMENT_LIBRARY).string()
: (rsrc_vendor_dir / PresetBundle::ORCA_FILAMENT_LIBRARY).string();
}
void GuideFrame::on_profile_loaded()
{
// Must be called on the main thread.
SaveProfileData();
const std::string strAll = m_ProfileJson.dump(-1, ' ', false, json::error_handler_t::ignore);
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << ", finished, json contents:\n" << strAll;
json res;
res["command"] = "userguide_profile_load_finish";
res["sequence_id"] = "10001";
RunScript(wxString::Format("HandleStudio(%s)", res.dump(-1, ' ', true)));
}
void GuideFrame::OnNavigationComplete(wxWebViewEvent &evt)
{
//wxLogMessage("%s", "Navigation complete; url='" + evt.GetURL() + "'");
if (!bFirstComplete) {
m_load_task = new boost::thread(boost::bind(&GuideFrame::LoadProfileData, this));
// boost::thread LoadProfileThread(boost::bind(&GuideFrame::LoadProfileData, this));
//LoadProfileThread.detach();
bFirstComplete = true;
try {
init_guide_paths();
if (BuildProfileDataFromPresetBundle()) {
if (!*m_cancel_token)
on_profile_loaded();
} else {
// Presets not yet in memory — delegate to background thread.
m_load_task = std::make_unique<boost::thread>(boost::bind(&GuideFrame::LoadProfileData, this));
}
} catch (const std::exception& e) {
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << ", init error: " << e.what();
m_load_task = std::make_unique<boost::thread>(boost::bind(&GuideFrame::LoadProfileData, this));
}
}
m_browser->Show();
@@ -762,11 +817,9 @@ bool GuideFrame::apply_config(AppConfig *app_config, PresetBundle *preset_bundle
bool check_unsaved_preset_changes = false;
std::vector<std::string> install_bundles;
std::vector<std::string> remove_bundles;
const auto vendor_dir = (boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR).make_preferred();
for (const auto &it : enabled_vendors) {
if (it.second.size() > 0) {
auto vendor_file = vendor_dir/(it.first + ".json");
if (!fs::exists(vendor_file)) {
if (!is_vendor_installed(it.first)) {
install_bundles.emplace_back(it.first);
}
}
@@ -777,8 +830,7 @@ bool GuideFrame::apply_config(AppConfig *app_config, PresetBundle *preset_bundle
if (it.second.size() > 0) {
if (enabled_vendors.find(it.first) != enabled_vendors.end())
continue;
auto vendor_file = vendor_dir/(it.first + ".json");
if (fs::exists(vendor_file)) {
if (is_vendor_installed(it.first)) {
remove_bundles.emplace_back(it.first);
}
}
@@ -1127,99 +1179,244 @@ int GuideFrame::GetFilamentInfo( std::string VendorDirectory, json & pFilaList,
return status;
}
int GuideFrame::LoadProfileData()
bool GuideFrame::BuildProfileJson(const PresetBundle& bundle, bool require_all_resource_vendors)
{
try {
m_ProfileJson = json::parse("{}");
m_ProfileJson["model"] = json::array();
m_ProfileJson["machine"] = json::object();
m_ProfileJson["filament"] = json::object();
m_ProfileJson["process"] = json::array();
// Models from vendor profiles
for (const auto& [vendor_id, vp] : bundle.vendors) {
for (const auto& model : vp.models) {
std::string nozzle_str;
for (const auto& v : model.variants) {
if (!nozzle_str.empty()) nozzle_str += ";";
nozzle_str += v.name;
}
const std::string materials_str = boost::algorithm::join(model.default_materials, ";");
boost::filesystem::path cover_path =
(boost::filesystem::path(resources_dir()) / "profiles" / vp.id / (model.id + "_cover.png"))
.make_preferred();
if (!boost::filesystem::exists(cover_path))
cover_path =
(boost::filesystem::path(resources_dir()) / "web/image/printer" / (model.id + "_cover.png"))
.make_preferred();
vendor_dir = (boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR).make_preferred();
rsrc_vendor_dir = (boost::filesystem::path(resources_dir()) / "profiles").make_preferred();
// Orca: add custom as default
// Orca: add json logic for vendor bundle
orca_bundle_rsrc = true;
// search if there exists a .json file in vendor_dir folder, if exists, set orca_bundle_rsrc to false
for (const auto& entry : boost::filesystem::directory_iterator(vendor_dir)) {
if (!boost::filesystem::is_directory(entry) && boost::iequals(entry.path().extension().string(), ".json") && !boost::iequals(entry.path().stem().string(), PresetBundle::ORCA_FILAMENT_LIBRARY)) {
orca_bundle_rsrc = false;
break;
json entry;
entry["model"] = model.id;
entry["name"] = model.name;
entry["vendor"] = vp.id;
entry["nozzle_diameter"] = nozzle_str;
entry["materials"] = materials_str;
entry["cover"] = cover_path.string();
entry["nozzle_selected"] = "";
entry["sub_path"] = "";
m_ProfileJson["model"].push_back(entry);
}
}
// load the default filament library first
std::set<std::string> loaded_vendors;
auto filament_library_name = boost::filesystem::path(PresetBundle::ORCA_FILAMENT_LIBRARY).replace_extension(".json");
if (boost::filesystem::exists(vendor_dir / filament_library_name)) {
m_OrcaFilaLibPath = (vendor_dir / PresetBundle::ORCA_FILAMENT_LIBRARY).string();
LoadProfileFamily(PresetBundle::ORCA_FILAMENT_LIBRARY, (vendor_dir / filament_library_name).string());
} else {
m_OrcaFilaLibPath = (rsrc_vendor_dir / PresetBundle::ORCA_FILAMENT_LIBRARY).string();
LoadProfileFamily(PresetBundle::ORCA_FILAMENT_LIBRARY, (rsrc_vendor_dir / filament_library_name).string());
}
loaded_vendors.insert(PresetBundle::ORCA_FILAMENT_LIBRARY);
// Machine map: preset name -> {model, nozzle variant}
for (const Preset& p : bundle.printers()) {
if (!p.is_system || !p.vendor) continue;
const auto* printer_model = p.config.option<ConfigOptionString>("printer_model");
const auto* printer_variant = p.config.option<ConfigOptionString>("printer_variant");
if (!printer_model || printer_model->value.empty() || !printer_variant) continue;
//load custom bundle from user data path
boost::filesystem::directory_iterator endIter;
for (boost::filesystem::directory_iterator iter(vendor_dir); iter != endIter; iter++) {
if (!boost::filesystem::is_directory(*iter)) {
wxString strVendor = from_u8(iter->path().string()).BeforeLast('.');
strVendor = strVendor.AfterLast('\\');
strVendor = strVendor.AfterLast('/');
wxString strExtension = from_u8(iter->path().string()).AfterLast('.').Lower();
if(strExtension.CmpNoCase("json") != 0 || loaded_vendors.find(w2s(strVendor)) != loaded_vendors.end())
continue;
LoadProfileFamily(w2s(strVendor), iter->path().string());
loaded_vendors.insert(w2s(strVendor));
}
if (m_destroy)
return 0;
json mach;
mach["model"] = printer_model->value;
mach["nozzle"] = printer_variant->value;
m_ProfileJson["machine"][p.name] = mach;
}
boost::filesystem::directory_iterator others_endIter;
for (boost::filesystem::directory_iterator iter(rsrc_vendor_dir); iter != others_endIter; iter++) {
if (!boost::filesystem::is_directory(*iter)) {
wxString strVendor = from_u8(iter->path().string()).BeforeLast('.');
strVendor = strVendor.AfterLast('\\');
strVendor = strVendor.AfterLast('/');
wxString strExtension = from_u8(iter->path().string()).AfterLast('.').Lower();
if (strExtension.CmpNoCase("json") != 0 || loaded_vendors.find(w2s(strVendor)) != loaded_vendors.end())
continue;
// Filament map from system filament presets (vendor/type already resolved in config)
const json& machines = m_ProfileJson["machine"];
for (const Preset& p : bundle.filaments()) {
if (!p.is_system || !p.vendor) continue;
const auto* fila_vendor = p.config.option<ConfigOptionStrings>("filament_vendor");
const auto* fila_type = p.config.option<ConfigOptionStrings>("filament_type");
const auto* compat_printers = p.config.option<ConfigOptionStrings>("compatible_printers");
LoadProfileFamily(w2s(strVendor), iter->path().string());
loaded_vendors.insert(w2s(strVendor));
std::string vendor = (fila_vendor && !fila_vendor->values.empty()) ? fila_vendor->values[0] : "";
std::string type = (fila_type && !fila_type->values.empty()) ? fila_type->values[0] : "";
std::string model_list;
if (compat_printers) {
for (const std::string& pname : compat_printers->values) {
auto it = machines.find(pname);
if (it != machines.end()) {
const std::string m = (*it)["model"];
const std::string n = (*it)["nozzle"];
model_list += "[" + m + "++" + n + "]";
}
}
}
if (m_destroy)
return 0;
json ff;
ff["name"] = p.name;
ff["sub_path"] = p.file;
ff["vendor"] = vendor;
ff["type"] = type;
ff["models"] = model_list;
ff["selected"] = 0;
m_ProfileJson["filament"][p.name] = ff;
}
wxGetApp().CallAfter([this] {
if (!m_destroy) {
//sync to appconfig first to populate current selections
SaveProfileData();
// Process list from visible system print presets
for (const Preset& p : bundle.prints()) {
if (!p.is_system || !p.vendor || !p.is_visible) continue;
json entry;
entry["name"] = p.name;
entry["sub_path"] = p.file;
m_ProfileJson["process"].push_back(entry);
}
//sync to web after selections are populated
std::string strAll = m_ProfileJson.dump(-1, ' ', false, json::error_handler_t::ignore);
if (require_all_resource_vendors) {
// If rsrc_vendor_dir has vendors (profile JSONs, or the preset caches a
// packaged build ships instead) not covered by the current bundle, the
// bundle is incomplete (e.g. dev env where data_dir/system only has
// OrcaFilamentLibrary+Custom). Fall back so the slow path reads both dirs.
try {
for (const std::string& name : vendor_names_in(rsrc_vendor_dir)) {
if (bundle.vendors.find(name) == bundle.vendors.end()) {
BOOST_LOG_TRIVIAL(info) << "GuideFrame: vendor '" << name
<< "' in resources but not in preset_bundle — falling back to JSON loading";
reset_profile_json();
return false;
}
}
} catch (const std::exception&) {}
}
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << ", finished, json contents: " << std::endl << strAll;
json m_Res = json::object();
m_Res["command"] = "userguide_profile_load_finish";
m_Res["sequence_id"] = "10001";
wxString strJS = wxString::Format("HandleStudio(%s)", m_Res.dump(-1, ' ', true));
BOOST_LOG_TRIVIAL(info) << "GuideFrame: built profile data ("
<< m_ProfileJson["model"].size() << " models, "
<< m_ProfileJson["machine"].size() << " machines, "
<< m_ProfileJson["filament"].size() << " filaments)";
return !m_ProfileJson["machine"].empty();
} catch (const std::exception& e) {
BOOST_LOG_TRIVIAL(warning) << "GuideFrame::BuildProfileJson failed: " << e.what()
<< " — falling back to JSON loading";
reset_profile_json();
return false;
}
}
RunScript(strJS);
bool GuideFrame::BuildProfileDataFromPresetBundle()
{
PresetBundle* pb = wxGetApp().preset_bundle;
if (!pb || pb->vendors.empty())
return false;
return BuildProfileJson(*pb, /*require_all_resource_vendors=*/true);
}
bool GuideFrame::BuildProfileDataFromVendors()
{
try {
// Same vendor set and precedence as the JSON scan in LoadProfileData: a
// vendor in the user's system dir shadows the bundled one of that name.
// A vendor is named by its profile or, where a build ships preset caches
// instead, by its cache alone — so both forms name one here.
std::map<std::string, boost::filesystem::path> vendor_files;
auto collect = [&vendor_files](const boost::filesystem::path& dir) {
boost::system::error_code ec;
if (!boost::filesystem::exists(dir, ec))
return;
for (const auto& e : boost::filesystem::directory_iterator(dir, ec))
if (Slic3r::is_json_file(e.path().string()) || e.path().extension() == ".opc")
vendor_files.emplace(e.path().stem().string(), e.path()); // first wins
};
collect(vendor_dir);
collect(rsrc_vendor_dir);
// Each vendor comes from its preset cache where one covers it, which is what
// makes this worth doing instead of the scan below; the filament library goes
// first because the others' filaments inherit from it, and resolving those on
// the vendors the cache does not cover needs it already loaded.
PresetBundle bundle;
auto load_vendor = [this](PresetBundle& into, const std::string& vendor, const PresetBundle* base) {
into.load_vendor_configs_from_json(vendor_dir.string(), vendor, PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent, base);
};
const std::string filament_library(PresetBundle::ORCA_FILAMENT_LIBRARY);
if (vendor_files.count(filament_library))
load_vendor(bundle, filament_library, nullptr);
for (const auto& entry : vendor_files) {
if (*m_cancel_token)
return false; // as in the scan below: a vendor without a cache is parsed, and that takes time
const std::string& vendor = entry.first;
// A cache is only ever written for a versioned vendor; a JSON has to be
// asked, so that an unversioned one (blacklist.json) carrying no presets
// is passed over.
if (vendor == filament_library ||
(entry.second.extension() != ".opc" && get_vendor_cache_version(entry.second.string()).empty()))
continue;
PresetBundle tmp;
load_vendor(tmp, vendor, &bundle);
bundle.merge_presets(std::move(tmp));
}
if (bundle.vendors.empty())
return false;
return BuildProfileJson(bundle, /*require_all_resource_vendors=*/false);
} catch (const std::exception& e) {
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << " failed: " << e.what();
reset_profile_json();
return false;
}
}
int GuideFrame::LoadProfileData()
{
// Background thread: the fast path in OnNavigationComplete failed (presets not yet loaded).
// Loading order (fastest to slowest):
// 1. Load every vendor, from its preset cache wherever one covers it
// 2. Read all vendor JSONs by hand
try {
if (!BuildProfileDataFromVendors()) {
// Last resort — read all vendor JSONs
std::set<std::string> loaded_vendors;
auto filament_library_name = boost::filesystem::path(PresetBundle::ORCA_FILAMENT_LIBRARY).replace_extension(".json");
if (boost::filesystem::exists(vendor_dir / filament_library_name))
LoadProfileFamily(PresetBundle::ORCA_FILAMENT_LIBRARY, (vendor_dir / filament_library_name).string());
else
LoadProfileFamily(PresetBundle::ORCA_FILAMENT_LIBRARY, (rsrc_vendor_dir / filament_library_name).string());
loaded_vendors.insert(PresetBundle::ORCA_FILAMENT_LIBRARY);
boost::filesystem::directory_iterator endIter;
for (boost::filesystem::directory_iterator iter(vendor_dir); iter != endIter; iter++) {
if (!boost::filesystem::is_directory(*iter)) {
wxString strVendor = from_u8(iter->path().string()).BeforeLast('.');
strVendor = strVendor.AfterLast('\\');
strVendor = strVendor.AfterLast('/');
wxString strExtension = from_u8(iter->path().string()).AfterLast('.').Lower();
if (strExtension.CmpNoCase("json") != 0 || loaded_vendors.find(w2s(strVendor)) != loaded_vendors.end())
continue;
LoadProfileFamily(w2s(strVendor), iter->path().string());
loaded_vendors.insert(w2s(strVendor));
}
if (*m_cancel_token) return 0;
}
boost::filesystem::directory_iterator others_endIter;
for (boost::filesystem::directory_iterator iter(rsrc_vendor_dir); iter != others_endIter; iter++) {
if (!boost::filesystem::is_directory(*iter)) {
wxString strVendor = from_u8(iter->path().string()).BeforeLast('.');
strVendor = strVendor.AfterLast('\\');
strVendor = strVendor.AfterLast('/');
wxString strExtension = from_u8(iter->path().string()).AfterLast('.').Lower();
if (strExtension.CmpNoCase("json") != 0 || loaded_vendors.find(w2s(strVendor)) != loaded_vendors.end())
continue;
LoadProfileFamily(w2s(strVendor), iter->path().string());
loaded_vendors.insert(w2s(strVendor));
}
if (*m_cancel_token) return 0;
}
}
// Capture the cancel token by value (shared_ptr) so the lambda doesn't
// touch `this` if GuideFrame is destroyed before the event fires.
auto tok = m_cancel_token;
wxGetApp().CallAfter([this, tok] {
if (!*tok)
on_profile_loaded();
});
} catch (std::exception& e) {
// wxLogMessage("GUIDE: load_profile_error %s ", e.what());
// wxMessageBox(e.what(), "", MB_OK);
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << ", error: " << e.what() << std::endl;
} catch (const std::exception& e) {
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << ", error: " << e.what();
}
filament_info_cache.clear();

View File

@@ -30,10 +30,14 @@
#include "libslic3r/PresetBundle.hpp"
#include "slic3r/Utils/PresetUpdater.hpp"
#include <atomic>
#include <memory>
#include <unordered_map>
#include <nlohmann/json.hpp>
#include <boost/thread.hpp>
namespace Slic3r { namespace GUI {
class GuideFrame : public DPIDialog
@@ -78,6 +82,12 @@ public:
int LoadProfileData();
int SaveProfileData();
int LoadProfileFamily(std::string strVendor, std::string strFilePath);
void init_guide_paths();
void on_profile_loaded();
bool BuildProfileJson(const PresetBundle& bundle, bool require_all_resource_vendors);
bool BuildProfileDataFromPresetBundle();
bool BuildProfileDataFromVendors();
void reset_profile_json();
int SaveProfile();
int GetFilamentInfo( std::string VendorDirectory,json & pFilaList, std::string filepath, std::string &sVendor, std::string &sType);
@@ -112,8 +122,11 @@ private:
//First Load
bool bFirstComplete{false};
bool m_destroy{false};
boost::thread* m_load_task{ nullptr };
// Set once in the destructor. Read through `this` by the loading thread
// (joined before `this` dies) and captured as the shared_ptr by CallAfter
// lambdas so they don't touch `this` after the object is freed.
std::shared_ptr<std::atomic<bool>> m_cancel_token{std::make_shared<std::atomic<bool>>(false)};
std::unique_ptr<boost::thread> m_load_task;
// User Config
bool PrivacyUse;
@@ -123,6 +136,7 @@ private:
bool InstallNetplugin;
bool network_plugin_ready {false};
json m_ProfileJson;
json m_OrcaFilaList;
std::string m_OrcaFilaLibPath;

View File

@@ -1042,47 +1042,43 @@ void PresetUpdater::priv::check_installed_vendor_profiles() const
std::set<std::string> bundles;
// Orca: always install filament library
bundles.insert(PresetBundle::ORCA_FILAMENT_LIBRARY);
for (auto &dir_entry : boost::filesystem::directory_iterator(rsrc_path)) {
const auto &path = dir_entry.path();
std::string file_path = path.string();
if (is_json_file(file_path)) {
const auto path_in_vendor = vendor_path / path.filename();
std::string vendor_name = path.filename().string();
// Remove the .json suffix.
vendor_name.erase(vendor_name.size() - 5);
if (bundles.find(vendor_name) != bundles.end())continue;
// A vendor is named by its profile or, where the build ships preset caches
// instead of the raw profile JSONs, by its cache alone.
for (const std::string &vendor_name : vendor_names_in(rsrc_path)) {
if (bundles.find(vendor_name) != bundles.end())continue;
const auto is_vendor_enabled = (vendor_name == PresetBundle::ORCA_DEFAULT_BUNDLE) // always update configs from resource to vendor for ORCA_DEFAULT_BUNDLE
|| (enabled_vendors.find(vendor_name) != enabled_vendors.end());
if (enabled_config_update) {
if ( fs::exists(path_in_vendor)) {
if (is_vendor_enabled) {
Semver resource_ver = get_version_from_json(file_path);
Semver vendor_ver = get_version_from_json(path_in_vendor.string());
const auto is_vendor_enabled = (vendor_name == PresetBundle::ORCA_DEFAULT_BUNDLE) // always update configs from resource to vendor for ORCA_DEFAULT_BUNDLE
|| (enabled_vendors.find(vendor_name) != enabled_vendors.end());
if (enabled_config_update) {
if (is_vendor_installed(vendor_name)) {
if (is_vendor_enabled) {
// Orca: whichever form of the vendor resources ships at the newer
// version is the one installing lays down, and the one to judge
// what is installed against.
Semver resource_ver = resource_vendor_version(vendor_name);
// Orca: a vendor installed as a preset cache has no profile
// beside it; the version it was installed at is in the cache.
Semver vendor_ver = installed_vendor_version(vendor_name);
bool version_match = ((resource_ver.maj() == vendor_ver.maj()) && (resource_ver.min() == vendor_ver.min()));
bool version_match = ((resource_ver.maj() == vendor_ver.maj()) && (resource_ver.min() == vendor_ver.min()));
if (!version_match || (vendor_ver < resource_ver)) {
BOOST_LOG_TRIVIAL(info) << "[Orca Updater]:found vendor "<<vendor_name<<" newer version "<<resource_ver.to_string() <<" from resource, old version "<<vendor_ver.to_string();
bundles.insert(vendor_name);
}
}
else {
//need to be removed because not installed
fs::remove(path_in_vendor);
const auto path_of_vendor = vendor_path / vendor_name;
if (fs::exists(path_of_vendor))
fs::remove_all(path_of_vendor);
if (!version_match || (vendor_ver < resource_ver)) {
BOOST_LOG_TRIVIAL(info) << "[Orca Updater]:found vendor "<<vendor_name<<" newer version "<<resource_ver.to_string() <<" from resource, old version "<<vendor_ver.to_string();
bundles.insert(vendor_name);
}
}
else if (is_vendor_enabled) {
bundles.insert(vendor_name);
else {
//need to be removed because not installed
remove_installed_vendor(vendor_name);
}
}
else if (is_vendor_enabled) {
bundles.insert(vendor_name);
}
}
else if (is_vendor_enabled) {
bundles.insert(vendor_name);
}
}
if (bundles.size() > 0) {
@@ -1162,11 +1158,12 @@ Updates PresetUpdater::priv::get_config_updates(const Semver &old_slic3r_version
auto filament_in_cache = (cache_profile_path / vendor_name / PRESET_FILAMENT_NAME);
auto machine_in_cache = (cache_profile_path / vendor_name / PRESET_PRINTER_NAME);
if (( fs::exists(path_in_vendor))
if (is_vendor_installed(vendor_name)
|| fs::exists(print_in_cache)
|| fs::exists(filament_in_cache)
|| fs::exists(machine_in_cache)) {
Semver vendor_ver = get_version_from_json(path_in_vendor.string());
// Orca: a vendor installed as a preset cache carries its version there.
Semver vendor_ver = installed_vendor_version(vendor_name);
std::map<std::string, std::string> key_values;
std::vector<std::string> keys(3);

View File

@@ -7,16 +7,8 @@
#include <boost/algorithm/string/predicate.hpp>
#include <boost/log/trivial.hpp>
#include <algorithm>
#include <cstdlib>
#include <future>
#include <slic3r/GUI/BindDialog.hpp>
#include <slic3r/GUI/GUI_App.hpp>
#include <slic3r/plugin/PluginFsUtils.hpp>
#include <slic3r/plugin/PluginManager.hpp>
#include <utility>
#include <wx/event.h>
#include <wx/msgdlg.h>
namespace Slic3r {
@@ -86,21 +78,24 @@ bool is_inside_allowed_root(const boost::filesystem::path& candidate, const boos
// ScopedPluginAuditContext
// ---------------------------------------------------------------------------
thread_local std::string PluginAuditManager::m_current_plugin_key = "";
thread_local std::string PluginAuditManager::m_current_capability_name = "";
thread_local std::string PluginAuditManager::m_current_plugin_key = "";
thread_local std::string PluginAuditManager::m_current_capability_name = "";
thread_local PluginAuditManager::AuditMode PluginAuditManager::m_audit_mode = PluginAuditManager::AuditMode::Loading;
thread_local std::vector<boost::filesystem::path> PluginAuditManager::m_scoped_allowed_roots;
thread_local bool PluginAuditManager::m_audit_denial_pending = false;
thread_local bool PluginAuditManager::m_has_last_violation = false;
thread_local AuditViolation PluginAuditManager::m_last_violation;
ScopedPluginAuditContext::ScopedPluginAuditContext(const std::string& plugin_key,
const std::string& capability_name)
const std::string& capability_name,
PluginAuditManager::AuditMode mode)
: m_previous_id(PluginAuditManager::instance().current_plugin())
, m_previous_capability(PluginAuditManager::instance().current_capability())
, m_previous_mode(PluginAuditManager::instance().audit_mode())
, m_previous_scoped_roots(PluginAuditManager::m_scoped_allowed_roots)
{
PluginAuditManager::instance().set_current_plugin(plugin_key);
PluginAuditManager::instance().set_current_capability(capability_name);
PluginAuditManager::instance().set_audit_mode(mode);
PluginAuditManager::m_scoped_allowed_roots.clear();
}
@@ -108,6 +103,7 @@ ScopedPluginAuditContext::~ScopedPluginAuditContext()
{
PluginAuditManager::instance().set_current_plugin(m_previous_id);
PluginAuditManager::instance().set_current_capability(m_previous_capability);
PluginAuditManager::instance().set_audit_mode(m_previous_mode);
PluginAuditManager::m_scoped_allowed_roots = std::move(m_previous_scoped_roots);
}
@@ -202,6 +198,14 @@ bool PluginAuditManager::is_denied_filename(const boost::filesystem::path& candi
return false;
}
// ---------------------------------------------------------------------------
// Audit mode
// ---------------------------------------------------------------------------
void PluginAuditManager::set_audit_mode(AuditMode mode) { m_audit_mode = mode; }
PluginAuditManager::AuditMode PluginAuditManager::audit_mode() const { return m_audit_mode; }
// ---------------------------------------------------------------------------
// Policy checks
// ---------------------------------------------------------------------------
@@ -215,15 +219,21 @@ AuditDecision PluginAuditManager::check_path_access(const boost::filesystem::pat
if (plugin_key.empty())
return {true, ""}; // not running inside a plugin context
// Denied filenames are checked before the allowed roots. The app config and the cloud
// refresh token live directly inside data_dir(), which is a global allowed root, so a deny
// placed any lower would be unreachable.
// Denied filenames are checked first, above both the Loading exemption below and the
// allowed roots. The app config and the cloud refresh token live directly inside
// data_dir(), which is a global allowed root, and no scope ever sets Enforcing — so a
// deny placed any lower would be unreachable for reads.
if (is_denied_filename(path)) {
BOOST_LOG_TRIVIAL(warning) << "[AUDIT] block path=" << path.string() << " is_write=" << is_write
<< " plugin=" << plugin_key << " reason=denied filename";
return {false, "denied filename"};
}
// During import/loading, only block writes. Python must be able to read
// stdlib modules and the plugin file itself during import.
if (m_audit_mode == AuditMode::Loading && !is_write)
return {true, ""};
namespace fs = boost::filesystem;
fs::path candidate = path;
@@ -251,6 +261,7 @@ AuditDecision PluginAuditManager::check_path_access(const boost::filesystem::pat
}
BOOST_LOG_TRIVIAL(warning) << "[AUDIT] block path=" << candidate.string() << " is_write=" << is_write
<< " audit_mode=" << (m_audit_mode == AuditMode::Loading ? "Loading" : "Enforcing")
<< " plugin=" << plugin_key;
return {false, "outside allowed root"};
}
@@ -258,99 +269,19 @@ AuditDecision PluginAuditManager::check_path_access(const boost::filesystem::pat
AuditDecision PluginAuditManager::check_open(const std::string& path_str, const std::string& mode)
{
const bool is_write = mode.find('w') != std::string::npos || mode.find('a') != std::string::npos ||
mode.find('+') != std::string::npos || mode.find('x') != std::string::npos;
mode.find('+') != std::string::npos;
return check_path_access(boost::filesystem::path(path_str), is_write);
}
bool PluginAuditManager::request_filesystem_read_permissions(const std::string& plugin_key,
const std::vector<std::string>& paths)
{
if (plugin_key.empty() || paths.empty())
return true;
PluginDescriptor descriptor;
if (!PluginManager::instance().try_get_plugin_descriptor(plugin_key, descriptor) || descriptor.plugin_root.empty())
return false;
PluginInstallState state;
read_install_state(boost::filesystem::path(descriptor.plugin_root), state);
std::vector<std::string> missing;
for (const std::string& path : paths) {
if (std::find(state.permissions.fs_read.begin(), state.permissions.fs_read.end(), path) == state.permissions.fs_read.end())
missing.push_back(path);
}
if (missing.empty())
return true;
if (wxTheApp == nullptr || GUI::wxGetApp().is_closing())
return false;
auto show_dialog = [&descriptor, &missing]() {
wxString requested_paths;
for (const std::string& path : missing)
requested_paths += wxString::FromUTF8(path.c_str()) + "\n";
wxMessageDialog dialog(
nullptr,
wxString::Format("Plugin \"%s\" requests filesystem read access to:\n%s",
wxString::FromUTF8(descriptor.name.c_str()), requested_paths),
"Plugin permissions",
wxYES_NO | wxICON_WARNING);
return dialog.ShowModal() == wxID_YES;
};
bool granted = false;
if (wxIsMainThread()) {
granted = show_dialog();
} else {
auto result = std::make_shared<std::promise<bool>>();
auto future = result->get_future();
GUI::wxGetApp().CallAfter([result, descriptor_name = descriptor.name, missing]() {
wxString requested_paths;
for (const std::string& path : missing)
requested_paths += wxString::FromUTF8(path.c_str()) + "\n";
wxMessageDialog dialog(
nullptr,
wxString::Format("Plugin \"%s\" requests filesystem read access to:\n%s",
wxString::FromUTF8(descriptor_name.c_str()), requested_paths),
"Plugin permissions",
wxYES_NO | wxICON_WARNING);
result->set_value(dialog.ShowModal() == wxID_YES);
});
granted = future.get();
}
if (!granted)
return false;
if (state.plugin_name.empty()) {
state.installed_from = descriptor.is_cloud_plugin() ? "cloud" : "local";
state.installed_version = !descriptor.installed_version.empty() ? descriptor.installed_version : descriptor.version;
state.plugin_name = descriptor.name;
state.cloud_uuid = descriptor.cloud_uuid();
state.enabled = true;
}
state.permissions.fs_read.insert(state.permissions.fs_read.end(), missing.begin(), missing.end());
return write_install_state(boost::filesystem::path(descriptor.plugin_root), state);
}
void PluginAuditManager::report_violation(const AuditViolation& violation)
{
m_last_violation = violation;
m_has_last_violation = true;
m_audit_denial_pending = true;
BOOST_LOG_TRIVIAL(warning) << "[AUDIT BLOCKED] plugin=" << violation.plugin_key << " event=" << violation.event_name
<< " path=" << violation.path.string() << " reason=" << violation.reason;
}
bool PluginAuditManager::audit_denial_pending() const { return m_audit_denial_pending; }
void PluginAuditManager::clear_audit_denial() { m_audit_denial_pending = false; }
void PluginAuditManager::clear_last_violation()
{
m_has_last_violation = false;
@@ -370,106 +301,27 @@ bool PluginAuditManager::last_violation(AuditViolation& violation) const
// The C-level audit hook
// ---------------------------------------------------------------------------
namespace PluginAuditDetail {
namespace {
PyObject* tuple_item(PyObject* args, Py_ssize_t index)
{
if (!args || !PyTuple_Check(args) || index < 0 || index >= PyTuple_GET_SIZE(args))
return nullptr;
return PyTuple_GET_ITEM(args, index);
}
std::string python_path(PyObject* object)
{
if (!object)
return {};
PyObject* path_object = PyOS_FSPath(object);
if (!path_object) {
PyErr_Clear();
return {};
}
const char* path = PyUnicode_Check(path_object) ? PyUnicode_AsUTF8(path_object) : PyBytes_AsString(path_object);
std::string result = path ? path : "";
Py_DECREF(path_object);
if (!path)
PyErr_Clear();
return result;
}
void add_filesystem_path(std::vector<boost::filesystem::path>& paths, PyObject* object)
{
const std::string path = python_path(object);
if (!path.empty())
paths.emplace_back(path);
}
std::vector<boost::filesystem::path> filesystem_paths(const std::string& event_name, PyObject* args)
{
std::vector<boost::filesystem::path> paths;
if (event_name == "open") {
add_filesystem_path(paths, tuple_item(args, 0));
} else if (event_name == "os.rename") {
add_filesystem_path(paths, tuple_item(args, 0));
add_filesystem_path(paths, tuple_item(args, 1));
} else if (event_name == "os.remove") {
add_filesystem_path(paths, tuple_item(args, 0));
}
return paths;
}
bool has_filesystem_permission(const PluginInstallState& state, const boost::filesystem::path& path)
{
return std::find(state.permissions.fs_read.begin(), state.permissions.fs_read.end(), path.string()) !=
state.permissions.fs_read.end();
}
bool persist_filesystem_permission(const std::string& plugin_key,
PluginInstallState& state,
const boost::filesystem::path& path)
{
PluginDescriptor descriptor;
if (!PluginManager::instance().try_get_plugin_descriptor(plugin_key, descriptor) || descriptor.plugin_root.empty())
return false;
// A sandbox plugin may not have a sidecar yet. Seed the small amount of install metadata
// needed by the writer so clicking Yes still creates the JSON file.
if (state.plugin_name.empty()) {
state.installed_from = descriptor.is_cloud_plugin() ? "cloud" : "local";
state.installed_version = !descriptor.installed_version.empty() ? descriptor.installed_version : descriptor.version;
state.plugin_name = descriptor.name;
state.cloud_uuid = descriptor.cloud_uuid();
state.enabled = true;
}
const std::string path_string = path.string();
if (std::find(state.permissions.fs_read.begin(), state.permissions.fs_read.end(), path_string) ==
state.permissions.fs_read.end())
state.permissions.fs_read.push_back(path_string);
return write_install_state(boost::filesystem::path(descriptor.plugin_root), state);
}
// Records a blocked event and raises PermissionError in the calling interpreter.
// Records a blocked event and raises PermissionError in the calling interpreter. Returns -1
// so an event branch can `return report_denied(...)` directly.
int report_denied(PluginAuditManager& mgr,
const std::string& event_name,
const boost::filesystem::path& target,
const boost::filesystem::path& path,
const AuditDecision& decision)
{
AuditViolation violation;
violation.plugin_key = mgr.current_plugin();
violation.event_name = event_name;
violation.path = target;
violation.path = path;
violation.reason = decision.reason;
mgr.report_violation(violation);
PyErr_SetString(PyExc_PermissionError, "Plugin attempted an audited operation without permission");
PyErr_SetString(PyExc_PermissionError, "Plugin attempted to access a blocked file path");
return -1;
}
} // namespace PluginAuditDetail
} // namespace
int PluginAuditManager::audit_hook(const char* event, PyObject* args, void* user_data)
{
@@ -484,53 +336,73 @@ int PluginAuditManager::audit_hook(const char* event, PyObject* args, void* user
BOOST_LOG_TRIVIAL(debug) << "[AUDIT EVENT] " << event_name;
}
if (mgr->current_plugin().empty())
return 0;
// extensive list of audit events can be found at https://docs.python.org/3/library/audit_events.html
PluginInstallState state;
const bool have_install_state = PluginManager::instance().get_install_state(mgr->current_plugin(), state);
if (!have_install_state) {
BOOST_LOG_TRIVIAL(warning) << __FUNCTION__ << " Failed to get install state for " << mgr->current_plugin();
// --- open event ---
if (event_name == "open") {
const char* path_cstr = nullptr;
const char* mode_cstr = nullptr;
int flags = 0;
// open(path, mode, flags) — path may be str, bytes, or int fd
if (!PyArg_ParseTuple(args, "s|si", &path_cstr, &mode_cstr, &flags)) {
PyErr_Clear(); // couldn't parse; allow
return 0;
}
std::string path_str(path_cstr ? path_cstr : "");
std::string mode_str(mode_cstr ? mode_cstr : "r");
AuditDecision decision = mgr->check_open(path_str, mode_str);
if (!decision.allowed)
return report_denied(*mgr, event_name, path_str, decision);
return 0;
}
const std::string plugin_name = state.plugin_name.empty() ? mgr->current_plugin() : state.plugin_name;
const std::vector<boost::filesystem::path> paths = PluginAuditDetail::filesystem_paths(event_name, args);
for (const auto& path : paths) {
if (PluginAuditDetail::has_filesystem_permission(state, path))
continue;
// --- os.rename event (raised by os.rename and os.replace) ---
if (event_name == "os.rename") {
const char* src_cstr = nullptr;
const char* dst_cstr = nullptr;
PyObject* src_dir_fd = nullptr;
PyObject* dst_dir_fd = nullptr;
wxMessageDialog dialog(
nullptr,
wxString::Format(_L("Plugin \"%s\" is requesting filesystem access to:\n%s"),
wxString::FromUTF8(plugin_name.c_str()),
wxString::FromUTF8(path.string().c_str())),
_L("Plugin permission request"),
wxYES_NO | wxICON_WARNING);
if (dialog.ShowModal() == wxID_YES &&
PluginAuditDetail::persist_filesystem_permission(mgr->current_plugin(), state, path))
continue;
// os.rename(src, dst, src_dir_fd, dst_dir_fd) — paths may be str, bytes, or int fd.
// The dir_fd arguments are unused, but must be accepted for the tuple to parse.
if (!PyArg_ParseTuple(args, "ss|OO", &src_cstr, &dst_cstr, &src_dir_fd, &dst_dir_fd)) {
PyErr_Clear(); // couldn't parse; allow
return 0;
}
return PluginAuditDetail::report_denied(
*mgr, event_name, path, {false, "filesystem permission required"});
// A rename writes at both ends, so either end being denied blocks the call.
for (const char* path_cstr : {src_cstr, dst_cstr}) {
std::string path_str(path_cstr ? path_cstr : "");
AuditDecision decision = mgr->check_path_access(path_str, /* is_write */ true);
if (!decision.allowed)
return report_denied(*mgr, event_name, path_str, decision);
}
return 0;
}
if (!paths.empty())
return 0;
// --- os.remove event (raised by os.remove and os.unlink) ---
if (event_name == "os.remove") {
const char* path_cstr = nullptr;
PyObject* dir_fd = nullptr;
wxMessageDialog dialog(
nullptr,
wxString::Format(
_L("Plugin \"%s\" is requesting permission for the Python audit event \"%s\".\n\n"
"This operation does not expose a filesystem path to the audit hook."),
wxString::FromUTF8(plugin_name.c_str()),
wxString::FromUTF8(event_name.c_str())),
_L("Plugin permission request"),
wxYES_NO | wxICON_WARNING);
if (dialog.ShowModal() == wxID_YES)
return 0;
// os.remove(path, dir_fd) — path may be str, bytes, or int fd
if (!PyArg_ParseTuple(args, "s|O", &path_cstr, &dir_fd)) {
PyErr_Clear(); // couldn't parse; allow
return 0;
}
return PluginAuditDetail::report_denied(
*mgr, event_name, boost::filesystem::path(), {false, "audit permission required"});
std::string path_str(path_cstr ? path_cstr : "");
AuditDecision decision = mgr->check_path_access(path_str, /* is_write */ true);
if (!decision.allowed)
return report_denied(*mgr, event_name, path_str, decision);
return 0;
}
// Unknown event — allow by default
return 0;
}
void PluginAuditManager::install_hook()
@@ -541,10 +413,10 @@ void PluginAuditManager::install_hook()
}
BOOST_LOG_TRIVIAL(info) << "[AUDIT] CPython audit hook installed successfully";
// data_dir() is the only globally-allowed root during plugin execution.
// data_dir() is the only globally-allowed root during enforced plugin execution.
// The executable directory and resources directory are intentionally NOT allowed
// here: plugins must not access outside data_dir() (G-code plugins additionally get
// the temp G-code folder via a scoped root).
// here: plugins must not write outside data_dir() (G-code plugins additionally get
// the temp G-code folder via a scoped root). Reads remain permissive in Loading mode.
add_global_allowed_root(data_dir());
// The user's app config and cloud credentials live directly inside data_dir(), so the

View File

@@ -2,7 +2,6 @@
#define slic3r_PluginAuditManager_hpp_
#include <Python.h>
#include <memory>
#include <mutex>
#include <string>
#include <vector>
@@ -54,8 +53,8 @@ public:
void add_scoped_allowed_root(const boost::filesystem::path& root);
// --- denied-filenames registry ---
// Filenames a plugin may never touch, in any directory, regardless of the enclosing allowed
// root. A candidate is denied when its filename starts with a
// Filenames a plugin may never touch, in any directory, regardless of audit mode or
// enclosing allowed root. A candidate is denied when its filename starts with a
// registered name, so .bak/.tmp companions are covered by the same entry.
//
// The comparison is case-insensitive on every platform, unlike the _WIN32-only iequals
@@ -74,23 +73,30 @@ public:
// 8.3 short name is out of scope (see the design doc). This blocks direct access only.
bool is_denied_filename(const boost::filesystem::path& candidate) const;
// --- enforcement mode ---
enum class AuditMode {
// Import/loading phase: allow reads anywhere, only block writes
// outside allowed roots. Python needs to read stdlib modules
// during import and those are not inside plugin directories.
Loading,
// Execution phase: block both reads and writes outside allowed
// roots, plus subprocess/socket/ctypes.
Enforcing,
};
void set_audit_mode(AuditMode mode);
AuditMode audit_mode() const;
// --- policy checks ---
// Shared core for every audited filesystem event. The deny list is consulted above the
// allowed roots, so a denied filename is blocked even when the file sits inside data_dir(),
// which is itself a global allowed root.
// Loading-mode read exemption and above the allowed roots, so a denied filename is
// blocked even though every scope currently runs in Loading and the files in question
// sit inside data_dir(), which is itself a global allowed root.
AuditDecision check_path_access(const boost::filesystem::path& candidate, bool is_write);
AuditDecision check_open(const std::string& path, const std::string& mode);
// Ask the user to grant the requested filesystem-read paths. The request may originate on a
// plugin load worker, so the implementation marshals the modal dialog to the wx main thread.
// Returns true only when every missing path was granted and persisted; denial aborts the plugin
// load without adding a permission.
bool request_filesystem_read_permissions(const std::string& plugin_key,
const std::vector<std::string>& paths);
void report_violation(const AuditViolation& violation);
bool audit_denial_pending() const;
void clear_audit_denial();
void clear_last_violation();
bool last_violation(AuditViolation& violation) const;
@@ -103,12 +109,12 @@ private:
static int audit_hook(const char* event, PyObject* args, void* user_data);
static thread_local std::string m_current_plugin_key;
static thread_local std::string m_current_capability_name;
static thread_local std::string m_current_plugin_key;
static thread_local std::string m_current_capability_name;
static thread_local AuditMode m_audit_mode;
static thread_local std::vector<boost::filesystem::path> m_scoped_allowed_roots;
static thread_local bool m_audit_denial_pending;
static thread_local bool m_has_last_violation;
static thread_local AuditViolation m_last_violation;
static thread_local bool m_has_last_violation;
static thread_local AuditViolation m_last_violation;
// mutable: is_denied_filename() is a const query that must lock.
mutable std::mutex m_mutex;
@@ -124,7 +130,8 @@ class ScopedPluginAuditContext
public:
explicit ScopedPluginAuditContext(
const std::string& plugin_key,
const std::string& capability_name = {});
const std::string& capability_name = {},
PluginAuditManager::AuditMode mode = PluginAuditManager::AuditMode::Loading);
~ScopedPluginAuditContext();
@@ -134,6 +141,7 @@ public:
private:
std::string m_previous_id;
std::string m_previous_capability;
PluginAuditManager::AuditMode m_previous_mode;
std::vector<boost::filesystem::path> m_previous_scoped_roots;
};

View File

@@ -785,23 +785,6 @@ bool read_install_state(const boost::filesystem::path& plugin_dir, PluginInstall
parsed.plugin_name = state["plugin_name"].get<std::string>();
if (state.contains("cloud_uuid") && state["cloud_uuid"].is_string())
parsed.cloud_uuid = state["cloud_uuid"].get<std::string>();
if (state.contains("permissions") && state["permissions"].is_object()) {
const auto& permissions = state["permissions"];
if (permissions.contains("networking") && permissions["networking"].is_array())
for (const auto& host : permissions["networking"])
if (host.is_string())
parsed.permissions.networking.push_back(host.get<std::string>());
if (permissions.contains("fs_read") && permissions["fs_read"].is_array())
for (const auto& path : permissions["fs_read"])
if (path.is_string())
parsed.permissions.fs_read.push_back(path.get<std::string>());
if (permissions.contains("fs_write") && permissions["fs_write"].is_array())
for (const auto& path : permissions["fs_write"])
if (path.is_string())
parsed.permissions.fs_write.push_back(path.get<std::string>());
}
if (state.contains("enabled") && state["enabled"].is_boolean())
parsed.enabled = state["enabled"].get<bool>();
@@ -837,12 +820,6 @@ bool write_install_state(const boost::filesystem::path& plugin_dir, const Plugin
if (!state.cloud_uuid.empty())
json["cloud_uuid"] = state.cloud_uuid;
json["permissions"] = {
{"networking", state.permissions.networking},
{"fs_read", state.permissions.fs_read},
{"fs_write", state.permissions.fs_write},
};
nlohmann::json capabilities = nlohmann::json::array();
for (const auto& [name, enabled] : state.capabilities)
capabilities.push_back(nlohmann::json{{name, enabled}});
@@ -860,9 +837,6 @@ bool write_install_state(const boost::filesystem::path& plugin_dir, const Plugin
const std::vector<std::pair<std::string, bool>>& capabilities)
{
PluginInstallState state;
// Loading a plugin updates its lifecycle/capability state, but must retain permissions granted
// during register_capabilities() or by a previous runtime audit prompt.
read_install_state(plugin_dir, state);
state.installed_from = entry.is_cloud_plugin() ? "cloud" : "local";
// Prefer the descriptor's recorded installed_version (the version fetched from the cloud
// at install time, preserved across sidecar re-writes) so a stale manifest/PEP723 header
@@ -879,16 +853,10 @@ bool write_install_state(const boost::filesystem::path& plugin_dir, const Plugin
bool write_install_state(const boost::filesystem::path& plugin_dir, const PluginDescriptor& entry)
{
// Install-time writer: the package is not loaded, so its capabilities are not known yet and the
// sidecar is (re)initialized to "auto-load, nothing disabled". This intentionally resets
// permissions on a fresh install/reinstall. PluginManager writes the real per-capability flags
// while preserving permissions once the package is loaded, via the overload above.
PluginInstallState state;
state.installed_from = entry.is_cloud_plugin() ? "cloud" : "local";
state.installed_version = !entry.installed_version.empty() ? entry.installed_version : entry.version;
state.plugin_name = entry.name;
state.cloud_uuid = entry.cloud_uuid();
state.enabled = true;
return write_install_state(plugin_dir, state);
// sidecar is (re)initialized to "auto-load, nothing disabled". PluginManager writes the real
// per-capability flags once the package is loaded, via the (dir, entry, enabled, capabilities)
// overload.
return write_install_state(plugin_dir, entry, true, {});
}
bool read_python_plugin_metadata(const boost::filesystem::path& py_path, PluginDescriptor& descriptor, std::string& error)

View File

@@ -81,21 +81,11 @@ inline nlohmann::json py_to_json(const pybind11::handle& o)
return py::str(o).cast<std::string>(); // fallback: str()
}
struct PluginPermissions
{
std::vector<std::string> networking;
std::vector<std::string> fs_read;
std::vector<std::string> fs_write;
};
struct PluginInstallState {
std::string installed_from; // "local" | "cloud"
std::string installed_version;
std::string plugin_name;
std::string cloud_uuid; // empty for local
PluginPermissions permissions;
bool enabled = true;
std::vector<std::pair<std::string, bool>> capabilities; // name -> enabled, ordered
};

View File

@@ -280,7 +280,7 @@ bool load(const PluginDescriptor& descriptor,
// (while the active plugin key is set), then instantiates each registered capability and caches
// its get_name(). Returns one entry per capability.
std::string bridge_error;
auto capabilities_found = bridge.finalize_plugin_capture(descriptor.entry_path, descriptor.plugin_key, bridge_error);
auto capabilities_found = bridge.finalize_plugin_capture(descriptor.entry_path, bridge_error);
if (!bridge_error.empty()) {
capabilities_found.clear();
error = "Plugin registration failed: " + bridge_error;

View File

@@ -21,7 +21,6 @@
#include <chrono>
#include <mutex>
#include <slic3r/plugin/PluginConfig.hpp>
#include <slic3r/plugin/PluginDescriptor.hpp>
#include <slic3r/plugin/PluginLoader.hpp>
#include <slic3r/plugin/PythonPluginInterface.hpp>
#include <slic3r/plugin/pluginTypes/script/ScriptPluginCapability.hpp>
@@ -555,20 +554,6 @@ std::shared_ptr<PluginCapabilityInterface> PluginManager::get_plugin_capability(
return nullptr;
}
bool PluginManager::get_install_state(const std::string& plugin_key, PluginInstallState& install_state)
{
PluginDescriptor descriptor;
if (!try_get_plugin_descriptor(plugin_key, descriptor)) {
return false;
}
if (!read_install_state(boost::filesystem::path(descriptor.plugin_root), install_state)) {
return false;
}
return true;
}
// ── Lifecycle ───────────────────────────────────────────────────────────────────────────────
bool PluginManager::is_plugin_loaded(const std::string& plugin_key) const
@@ -857,8 +842,6 @@ void PluginManager::load_plugin_impl(const std::string& plugin_key, bool skip_de
if (!plugin_loader::load(descriptor, skip_deps, capabilities_to_enable, registry_precheck, plugin, error)) {
if (error == LOAD_CANCELLED)
return; // cancelled: nothing materialized survives, and no error is recorded
if (error.rfind("Plugin registration failed:", 0) == 0)
mark_plugin_install_state_disabled(plugin_key);
fail(std::move(error));
return;
}
@@ -1084,31 +1067,6 @@ void PluginManager::write_loaded_plugin_install_state(const std::string& plugin_
write_install_state(boost::filesystem::path(descriptor.plugin_root), descriptor, /*enabled=*/true, capabilities);
}
void PluginManager::mark_plugin_install_state_disabled(const std::string& plugin_key)
{
std::lock_guard<std::mutex> state_lock(m_install_state_mutex);
PluginDescriptor descriptor;
if (!try_get_plugin_descriptor(plugin_key, descriptor) || descriptor.plugin_root.empty())
return;
const boost::filesystem::path root(descriptor.plugin_root);
PluginInstallState state;
if (!read_install_state(root, state)) {
state.installed_from = descriptor.is_cloud_plugin() ? "cloud" : "local";
state.installed_version = !descriptor.installed_version.empty() ? descriptor.installed_version : descriptor.version;
state.plugin_name = descriptor.name;
state.cloud_uuid = descriptor.cloud_uuid();
}
state.enabled = false;
if (!write_install_state(root, state))
return;
std::lock_guard<std::mutex> lock(m_mutex);
if (Plugin* plugin = find_plugin_locked(plugin_key))
plugin->descriptor.enabled = false;
}
// ── Callbacks ───────────────────────────────────────────────────────────────────────────────
void PluginManager::subscribe_on_load_callback(PluginLifecycleCompleteFn fn)

View File

@@ -150,8 +150,6 @@ public:
PluginCapabilityType type = PluginCapabilityType::Unknown,
bool only_enabled = true) const;
bool get_install_state(const std::string& plugin_key, PluginInstallState& install_state);
void load_plugin(const std::string& plugin_key, bool skip_deps = false, std::vector<std::string> capabilities_to_enable = {});
bool unload_plugin(const std::string& plugin_key);
void unload_all_plugins();
@@ -248,7 +246,6 @@ private:
// Writes the sidecar for a loaded plugin (enabled=true plus the current per-capability flags).
void write_loaded_plugin_install_state(const std::string& plugin_key);
void mark_plugin_install_state_disabled(const std::string& plugin_key);
bool finalize_cloud_plugin_removal(const PluginDescriptor& plugin, bool keep_local, std::string& error);
bool delete_installed_plugin_package(const PluginDescriptor& plugin, std::string& error);

View File

@@ -26,30 +26,25 @@
try { \
override_call; \
} catch (pybind11::error_already_set & err) { \
const bool _orca_audit_denial = ::Slic3r::PluginAuditManager::instance().audit_denial_pending(); \
if (_orca_audit_denial) \
::Slic3r::PluginAuditManager::instance().clear_current_plugin(); \
::Slic3r::log_python_exception_keep(err); \
if (_orca_audit_denial) \
::Slic3r::PluginAuditManager::instance().clear_audit_denial(); \
throw; \
}
// Opens the plugin's filesystem audit scope for the duration of a C++ -> Python call, and publishes
// the calling capability's cached name so host APIs invoked from Python can tell which capability
// they are serving. No-op without an audit plugin key. Declares a local `_orca_audit_scope`.
#define ORCA_PY_AUDIT_SCOPE() \
#define ORCA_PY_AUDIT_SCOPE(mode) \
std::optional<::Slic3r::ScopedPluginAuditContext> _orca_audit_scope; \
if (const std::string& _orca_audit_key = this->audit_plugin_key(); !_orca_audit_key.empty()) \
_orca_audit_scope.emplace(_orca_audit_key, this->name())
_orca_audit_scope.emplace(_orca_audit_key, this->name(), mode)
#define ORCA_PY_OVERRIDE_AUDITED(audit_setup, override_macro, ret, base, name, ...) \
#define ORCA_PY_OVERRIDE_AUDITED(mode, audit_setup, override_macro, ret, base, name, ...) \
do { \
::Slic3r::PluginCapabilityInterface::RefCounter _orca_ref_counter(*this); \
::Slic3r::PythonGILState _orca_python_gil; \
if (!_orca_python_gil) \
throw std::runtime_error("Python interpreter is shutting down"); \
ORCA_PY_AUDIT_SCOPE(); \
ORCA_PY_AUDIT_SCOPE(mode); \
if (_orca_audit_scope) \
audit_setup(); \
ORCA_PY_LOGGED_OVERRIDE_BODY(override_macro(ret, base, name, ##__VA_ARGS__)); \
@@ -63,7 +58,7 @@ public:
std::string get_name() const override
{
ORCA_PY_OVERRIDE_AUDITED([] {}, PYBIND11_OVERRIDE_PURE, std::string, Base, get_name);
ORCA_PY_OVERRIDE_AUDITED(::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, std::string, Base, get_name);
}
// Config UI hooks. Available on every capability type, so they live here rather than in
@@ -71,6 +66,7 @@ public:
bool has_config_ui() const override
{
ORCA_PY_OVERRIDE_AUDITED(
::Slic3r::PluginAuditManager::AuditMode::Loading,
[] {},
PYBIND11_OVERRIDE,
bool,
@@ -81,6 +77,7 @@ public:
std::string get_config_ui() const override
{
ORCA_PY_OVERRIDE_AUDITED(
::Slic3r::PluginAuditManager::AuditMode::Loading,
[] {},
PYBIND11_OVERRIDE,
std::string,
@@ -97,7 +94,7 @@ public:
// mistake), both fall back to the base's empty object rather than writing `"cap_config": null`.
nlohmann::json get_default_config() const override
{
ORCA_PY_AUDIT_SCOPE();
ORCA_PY_AUDIT_SCOPE(::Slic3r::PluginAuditManager::AuditMode::Loading);
try {
pybind11::gil_scoped_acquire gil;
pybind11::function override = pybind11::get_override(static_cast<const Base*>(this), "get_default_config");
@@ -120,17 +117,17 @@ public:
void on_load() override
{
ORCA_PY_OVERRIDE_AUDITED([] {}, PYBIND11_OVERRIDE, void, Base, on_load);
ORCA_PY_OVERRIDE_AUDITED(::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE, void, Base, on_load);
}
void on_unload() override
{
ORCA_PY_OVERRIDE_AUDITED([] {}, PYBIND11_OVERRIDE, void, Base, on_unload);
ORCA_PY_OVERRIDE_AUDITED(::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE, void, Base, on_unload);
}
void on_cancelled() override
{
ORCA_PY_OVERRIDE_AUDITED([] {}, PYBIND11_OVERRIDE, void, Base, on_cancelled);
ORCA_PY_OVERRIDE_AUDITED(::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE, void, Base, on_cancelled);
}
};
@@ -142,7 +139,7 @@ public:
PluginCapabilityType get_type() const override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE, PluginCapabilityType, PluginCapabilityInterface,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE, PluginCapabilityType, PluginCapabilityInterface,
get_type);
}
};

View File

@@ -87,12 +87,6 @@ void log_python_exception_keep(pybind11::error_already_set& err)
if (!gil)
return;
// Traceback output is host-owned work. In particular, the stderr tee opens the Python log
// file on every write. Keep that open outside the plugin audit context, otherwise an ordinary
// exception raised by a plugin can recursively trigger the filesystem permission dialog while
// its original exception is being reported.
ScopedPluginAuditContext audit_suppression("");
// Non-destructive: print the traceback to sys.stderr (tee'd to the session log)
// WITHOUT consuming err, so the caller can rethrow it intact. For example, downstream C++
// catchers can still read err.what() for the user-facing dialog. We must NOT use

View File

@@ -1,6 +1,5 @@
#include "PythonPluginBridge.hpp"
#include <algorithm>
#include <boost/log/trivial.hpp>
#include <exception>
#include <memory>
@@ -40,7 +39,6 @@ thread_local std::string g_active_plugin_key;
std::mutex g_registry_mutex;
std::unordered_map<std::string, std::vector<py::object>> g_pending_capabilities;
std::unordered_map<std::string, py::object> g_pending_package;
std::unordered_map<std::string, std::vector<std::string>> g_pending_fs_read_permissions;
struct PluginInstanceHandle
{
// The C++ plugin interface points into a Python object. Keep both alive through one
@@ -78,8 +76,6 @@ void discard_pending_capture_without_python(const std::string& plugin_key)
(void) package->second.release();
g_pending_package.erase(package);
}
g_pending_fs_read_permissions.erase(plugin_key);
}
} // namespace
@@ -104,38 +100,34 @@ void PythonPluginBridge::begin_plugin_capture(const std::string& plugin_key)
// for this same entry path.
g_pending_capabilities.erase(plugin_key);
g_pending_package.erase(plugin_key);
g_pending_fs_read_permissions.erase(plugin_key);
}
// From now until finalize/cancel, @orca.plugin and register_capability() calls made by
// Python code on this thread are attributed to this plugin.
g_active_plugin_key = plugin_key;
}
std::vector<CapturedCapability> PythonPluginBridge::finalize_plugin_capture(const std::string& capture_key,
const std::string& plugin_key,
std::string& error)
std::vector<CapturedCapability> PythonPluginBridge::finalize_plugin_capture(const std::string& plugin_key, std::string& error)
{
PythonGILState gil;
if (!gil) {
error = "Python interpreter is shutting down";
return {};
}
BOOST_LOG_TRIVIAL(info) << "Finalizing Python plugin capture for key " << capture_key;
BOOST_LOG_TRIVIAL(info) << "Finalizing Python plugin capture for key " << plugin_key;
// Phase 1: run the package class's register_capabilities() while the active key is
// still set. That method is expected to call orca.register_capability() once per
// capability class, and register_capability() needs g_active_plugin_key to know which
// pending bucket to append to.
{
auto clear_active_key = [&capture_key]() {
if (g_active_plugin_key == capture_key)
auto clear_active_key = [&plugin_key]() {
if (g_active_plugin_key == plugin_key)
g_active_plugin_key.clear();
};
auto discard_pending_for_key = [&capture_key]() {
auto discard_pending_for_key = [&plugin_key]() {
std::lock_guard<std::mutex> lock(g_registry_mutex);
g_pending_capabilities.erase(capture_key);
g_pending_package.erase(capture_key);
g_pending_fs_read_permissions.erase(capture_key);
g_pending_capabilities.erase(plugin_key);
g_pending_package.erase(plugin_key);
};
try {
@@ -145,7 +137,7 @@ std::vector<CapturedCapability> PythonPluginBridge::finalize_plugin_capture(cons
py::object package_cls;
{
std::lock_guard<std::mutex> lock(g_registry_mutex);
auto it = g_pending_package.find(capture_key);
auto it = g_pending_package.find(plugin_key);
if (it != g_pending_package.end()) {
package_cls = it->second;
g_pending_package.erase(it);
@@ -164,33 +156,6 @@ std::vector<CapturedCapability> PythonPluginBridge::finalize_plugin_capture(cons
// are kept.
py::object package = package_cls();
package.attr("register_capabilities")();
// Permission declarations are collected while register_capabilities() runs so the
// plugin can describe its needs without touching wx from the Python load worker. Ask
// only after registration returns, before capability instances are materialized.
std::vector<std::string> fs_read_permissions;
{
std::lock_guard<std::mutex> lock(g_registry_mutex);
auto it = g_pending_fs_read_permissions.find(capture_key);
if (it != g_pending_fs_read_permissions.end()) {
fs_read_permissions = std::move(it->second);
g_pending_fs_read_permissions.erase(it);
}
}
if (!fs_read_permissions.empty()) {
bool granted = false;
{
py::gil_scoped_release release;
granted = PluginAuditManager::instance().request_filesystem_read_permissions(plugin_key, fs_read_permissions);
}
if (!granted) {
error = "Plugin filesystem read permission request denied";
BOOST_LOG_TRIVIAL(warning) << error << " for key " << plugin_key;
discard_pending_for_key();
clear_active_key();
return {};
}
}
} catch (py::error_already_set& err) {
log_python_exception_keep(err);
error = err.what();
@@ -214,7 +179,7 @@ std::vector<CapturedCapability> PythonPluginBridge::finalize_plugin_capture(cons
std::vector<py::object> classes;
{
std::lock_guard<std::mutex> lock(g_registry_mutex);
auto it = g_pending_capabilities.find(capture_key);
auto it = g_pending_capabilities.find(plugin_key);
if (it != g_pending_capabilities.end()) {
classes = std::move(it->second);
g_pending_capabilities.erase(it);
@@ -223,10 +188,10 @@ std::vector<CapturedCapability> PythonPluginBridge::finalize_plugin_capture(cons
// Registration is complete. Later register_capability() calls should fail instead of
// accidentally attaching themselves to this plugin.
if (g_active_plugin_key == capture_key)
if (g_active_plugin_key == plugin_key)
g_active_plugin_key.clear();
BOOST_LOG_TRIVIAL(info) << "Collected " << classes.size() << " registered capability class(es) for key " << capture_key;
BOOST_LOG_TRIVIAL(info) << "Collected " << classes.size() << " registered capability class(es) for key " << plugin_key;
std::vector<CapturedCapability> capabilities;
capabilities.reserve(classes.size());
@@ -238,7 +203,7 @@ std::vector<CapturedCapability> PythonPluginBridge::finalize_plugin_capture(cons
py::object instance = cls();
if (!py::isinstance<PluginCapabilityInterface>(instance)) {
error = "Registered capability must inherit from a PluginCapability base";
BOOST_LOG_TRIVIAL(error) << "Python plugin capture failed type check for key " << capture_key
BOOST_LOG_TRIVIAL(error) << "Python plugin capture failed type check for key " << plugin_key
<< " error=" << error;
return {};
}
@@ -246,7 +211,7 @@ std::vector<CapturedCapability> PythonPluginBridge::finalize_plugin_capture(cons
auto capability_iface = instance.cast<std::shared_ptr<PluginCapabilityInterface>>();
if (!capability_iface) {
error = "Failed to cast Python capability to PluginCapabilityInterface";
BOOST_LOG_TRIVIAL(error) << "Python plugin capture failed cast for key " << capture_key
BOOST_LOG_TRIVIAL(error) << "Python plugin capture failed cast for key " << plugin_key
<< " error=" << error;
return {};
}
@@ -260,7 +225,7 @@ std::vector<CapturedCapability> PythonPluginBridge::finalize_plugin_capture(cons
// here is a hard error that rejects the whole plugin capture.
if (name.find(';') != std::string::npos) {
error = "Capability name must not contain ';': " + name;
BOOST_LOG_TRIVIAL(error) << "Python plugin capture rejected capability for key " << capture_key
BOOST_LOG_TRIVIAL(error) << "Python plugin capture rejected capability for key " << plugin_key
<< " error=" << error;
return {};
}
@@ -280,12 +245,12 @@ std::vector<CapturedCapability> PythonPluginBridge::finalize_plugin_capture(cons
// log the traceback here. GIL is held for the duration of finalize_plugin_capture.
log_python_exception_keep(err);
error = err.what();
BOOST_LOG_TRIVIAL(error) << "Python plugin capture raised Python exception for key " << capture_key
BOOST_LOG_TRIVIAL(error) << "Python plugin capture raised Python exception for key " << plugin_key
<< " error=" << error;
return {};
} catch (const std::exception& ex) {
error = ex.what();
BOOST_LOG_TRIVIAL(error) << "Python plugin capture raised exception for key " << capture_key
BOOST_LOG_TRIVIAL(error) << "Python plugin capture raised exception for key " << plugin_key
<< " error=" << error;
return {};
}
@@ -313,7 +278,6 @@ void PythonPluginBridge::cancel_plugin_capture(const std::string& plugin_key)
// may already have registered under this key.
g_pending_capabilities.erase(plugin_key);
g_pending_package.erase(plugin_key);
g_pending_fs_read_permissions.erase(plugin_key);
}
if (g_active_plugin_key == plugin_key)
@@ -340,7 +304,6 @@ void PythonPluginBridge::clear_pending_captures()
(void) pkg.release();
}
g_pending_package.clear();
g_pending_fs_read_permissions.clear();
g_active_plugin_key.clear();
return;
}
@@ -349,7 +312,6 @@ void PythonPluginBridge::clear_pending_captures()
BOOST_LOG_TRIVIAL(info) << "Clearing " << g_pending_capabilities.size() << " pending Python plugin capture(s)";
g_pending_capabilities.clear();
g_pending_package.clear();
g_pending_fs_read_permissions.clear();
g_active_plugin_key.clear();
}
@@ -486,27 +448,6 @@ void bind_python_api(pybind11::module_& m)
},
R"pbdoc(Register a PluginCapability subclass while OrcaSlicer loads your module.)pbdoc");
m.def(
"request_permissions",
[](const std::vector<std::string>& fs_read) {
if (g_active_plugin_key.empty())
throw py::value_error("request_permissions() called outside plugin discovery context");
std::lock_guard<std::mutex> lock(g_registry_mutex);
auto& requested = g_pending_fs_read_permissions[g_active_plugin_key];
for (const std::string& path : fs_read) {
if (path.empty())
throw py::value_error("request_permissions(fs_read=...) does not accept empty paths");
if (std::find(requested.begin(), requested.end(), path) == requested.end())
requested.push_back(path);
}
},
py::arg("fs_read") = std::vector<std::string>{},
R"pbdoc(Request filesystem read permissions while OrcaSlicer loads your module.
The host presents the request to the user after register_capabilities() returns. Denying the
request aborts the plugin load.)pbdoc");
m.def("plugin", [](py::object cls) {
if (g_active_plugin_key.empty())
throw py::value_error("@orca.plugin used outside plugin discovery context");

View File

@@ -31,7 +31,7 @@ public:
// Returns one CapturedCapability per capability, or an empty vector on failure
// (error message populated).
std::vector<CapturedCapability> finalize_plugin_capture(
const std::string& capture_key, const std::string& plugin_key, std::string& error);
const std::string& plugin_key, std::string& error);
// Clear any pending registrations for the key. Safe to call when import fails.
void cancel_plugin_capture(const std::string& plugin_key);

View File

@@ -16,206 +16,206 @@ public:
AgentInfo get_agent_info() override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, AgentInfo, PrinterAgentPluginCapability,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, AgentInfo, PrinterAgentPluginCapability,
get_agent_info);
}
int connect_printer(std::string dev_id, std::string dev_ip, std::string username, std::string password, bool use_ssl) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, connect_printer, dev_id,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, connect_printer, dev_id,
dev_ip, username, password, use_ssl);
}
int disconnect_printer() override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, disconnect_printer);
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, disconnect_printer);
}
int send_message(std::string dev_id, std::string json_str, int qos, int flag) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, send_message, dev_id,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, send_message, dev_id,
json_str, qos, flag);
}
int send_message_to_printer(std::string dev_id, std::string json_str, int qos, int flag) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, send_message_to_printer,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, send_message_to_printer,
dev_id, json_str, qos, flag);
}
bool start_discovery(bool start, bool sending) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, bool, PrinterAgentPluginCapability, start_discovery, start,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, bool, PrinterAgentPluginCapability, start_discovery, start,
sending);
}
int bind_detect(std::string dev_ip, std::string sec_link, detectResult& detect) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, bind_detect, dev_ip,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, bind_detect, dev_ip,
sec_link, detect);
}
std::string get_user_selected_machine() override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, std::string, PrinterAgentPluginCapability,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, std::string, PrinterAgentPluginCapability,
get_user_selected_machine);
}
int set_user_selected_machine(std::string dev_id) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability,
set_user_selected_machine, dev_id);
}
int start_send_gcode_to_sdcard(PrintParams params, OnUpdateStatusFn update_fn, WasCancelledFn cancel_fn, OnWaitFn wait_fn) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability,
start_send_gcode_to_sdcard, params, update_fn, cancel_fn, wait_fn);
}
int start_local_print(PrintParams params, OnUpdateStatusFn update_fn, WasCancelledFn cancel_fn) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, start_local_print,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, start_local_print,
params, update_fn, cancel_fn);
}
FilamentSyncMode get_filament_sync_mode() const override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, FilamentSyncMode, PrinterAgentPluginCapability,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, FilamentSyncMode, PrinterAgentPluginCapability,
get_filament_sync_mode);
}
bool fetch_filament_info(std::string dev_id) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, bool, PrinterAgentPluginCapability, fetch_filament_info, dev_id);
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, bool, PrinterAgentPluginCapability, fetch_filament_info, dev_id);
}
int check_cert() override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, check_cert);
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, check_cert);
}
void install_device_cert(std::string dev_id, bool lan_only) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, void, PrinterAgentPluginCapability, install_device_cert, dev_id,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, void, PrinterAgentPluginCapability, install_device_cert, dev_id,
lan_only);
}
int ping_bind(std::string ping_code) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, ping_bind, ping_code);
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, ping_bind, ping_code);
}
int bind(std::string dev_ip, std::string dev_id, std::string dev_model, std::string sec_link, std::string timezone, bool improved, OnUpdateStatusFn update_fn) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, bind, dev_ip, dev_id,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, bind, dev_ip, dev_id,
dev_model, sec_link, timezone, improved, update_fn);
}
int unbind(std::string dev_id) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, unbind, dev_id);
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, unbind, dev_id);
}
int start_print(PrintParams params, OnUpdateStatusFn update_fn, WasCancelledFn cancel_fn, OnWaitFn wait_fn) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, start_print, params,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, start_print, params,
update_fn, cancel_fn, wait_fn);
}
int start_local_print_with_record(PrintParams params, OnUpdateStatusFn update_fn, WasCancelledFn cancel_fn, OnWaitFn wait_fn) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability,
start_local_print_with_record, params, update_fn, cancel_fn, wait_fn);
}
int start_sdcard_print(PrintParams params, OnUpdateStatusFn update_fn, WasCancelledFn cancel_fn) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, start_sdcard_print, params,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, start_sdcard_print, params,
update_fn, cancel_fn);
}
int get_hms_snapshot(std::string dev_id, std::string file_name, std::function<void(std::string, int)> callback) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, get_hms_snapshot, dev_id,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, get_hms_snapshot, dev_id,
file_name, callback);
}
int set_server_callback(OnServerErrFn fn) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_server_callback, fn);
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_server_callback, fn);
}
int set_on_ssdp_msg_fn(OnMsgArrivedFn fn) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_on_ssdp_msg_fn, fn);
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_on_ssdp_msg_fn, fn);
}
int set_on_printer_connected_fn(OnPrinterConnectedFn fn) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_on_printer_connected_fn,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_on_printer_connected_fn,
fn);
}
int set_on_subscribe_failure_fn(GetSubscribeFailureFn fn) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_on_subscribe_failure_fn,
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_on_subscribe_failure_fn,
fn);
}
int set_on_message_fn(OnMessageFn fn) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_on_message_fn, fn);
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_on_message_fn, fn);
}
int set_on_user_message_fn(OnMessageFn fn) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_on_user_message_fn, fn);
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_on_user_message_fn, fn);
}
int set_on_local_connect_fn(OnLocalConnectedFn fn) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_on_local_connect_fn, fn);
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_on_local_connect_fn, fn);
}
int set_on_local_message_fn(OnMessageFn fn) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_on_local_message_fn, fn);
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_on_local_message_fn, fn);
}
int set_queue_on_main_fn(QueueOnMainFn fn) override
{
ORCA_PY_OVERRIDE_AUDITED(
[] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_queue_on_main_fn, fn);
::Slic3r::PluginAuditManager::AuditMode::Loading, [] {}, PYBIND11_OVERRIDE_PURE, int, PrinterAgentPluginCapability, set_queue_on_main_fn, fn);
}
// request_bind_ticket returns its ticket through a std::string* out-param, which pybind11
@@ -223,7 +223,7 @@ public:
// returns a (result, ticket) tuple, which we unpack into the int result and the out-param.
int request_bind_ticket(std::string* ticket) override
{
ORCA_PY_AUDIT_SCOPE();
ORCA_PY_AUDIT_SCOPE(::Slic3r::PluginAuditManager::AuditMode::Loading);
::Slic3r::PluginCapabilityInterface::RefCounter _orca_ref_counter(*this);
::Slic3r::PythonGILState gil;
if (!gil)

View File

@@ -13,6 +13,7 @@ public:
ExecutionResult execute() override
{
ORCA_PY_OVERRIDE_AUDITED(
::Slic3r::PluginAuditManager::AuditMode::Loading,
[] {},
PYBIND11_OVERRIDE_PURE,
ExecutionResult,

View File

@@ -10,6 +10,7 @@ public:
using PyPluginCommonTrampoline<SlicingPipelinePluginCapability>::PyPluginCommonTrampoline;
ExecutionResult execute(SlicingPipelineContext& ctx) override {
ORCA_PY_OVERRIDE_AUDITED(
::Slic3r::PluginAuditManager::AuditMode::Loading,
[&]{
// At Step.psGCodePostProcess the plugin edits the exported G-code file, which lives
// outside data_dir() (a temp/output folder), so writing to it would otherwise be

View File

@@ -153,6 +153,24 @@ TEST_CASE("Object brims are generated per instance", "[SkirtBrim]")
}
}
TEST_CASE("Uncombined neighboring brims precede their respective objects", "[SkirtBrim]")
{
Print print;
Model model;
place_two_cubes_apart(0, {
{ "skirt_loops", 0 },
{ "brim_type", "outer_only" },
{ "brim_width", 5 },
{ "combine_brims", 0 },
}, print, model);
print.process();
REQUIRE(print.skirt_brim_groups().size() == 1);
REQUIRE(print.skirt_brim_groups().front().brims.size() == 2);
CHECK(role_sequence(gcode(print), { "brim", "perimeter" }) ==
std::vector<std::string>{ "brim", "perimeter", "brim", "perimeter" });
}
TEST_CASE("Combine brims merges neighboring object instances", "[SkirtBrim]")
{
Print print;

View File

@@ -17,6 +17,7 @@ add_executable(${_TEST_NAME}_tests
test_preset_bundle_loading.cpp
test_preset_setting_id.cpp
test_preset_diff.cpp
test_vendor_cache.cpp
test_elephant_foot_compensation.cpp
test_geometry.cpp
test_multimaterial_segmentation.cpp

View File

@@ -146,7 +146,7 @@ TEST_CASE("Current vendor type tolerates missing printer model", "[Preset][Bundl
{
PresetBundle bundle;
VendorProfile orca_vendor("ORCA");
VendorProfile orca_vendor; orca_vendor.id = "ORCA";
VendorProfile::PrinterModel model;
model.name = "Orca Test";
orca_vendor.models.emplace_back(model);

File diff suppressed because it is too large Load Diff

View File

@@ -81,7 +81,7 @@ TEST_CASE("Plugin audit denies app config and token filenames anywhere", "[audit
}
}
TEST_CASE("Plugin audit deny beats allowed roots", "[audit]")
TEST_CASE("Plugin audit deny beats allowed roots and the Loading read exemption", "[audit]")
{
ScopedDataDir data_dir_guard("plugin-audit-deny");
seed_denied_names();
@@ -91,8 +91,8 @@ TEST_CASE("Plugin audit deny beats allowed roots", "[audit]")
// config and the token would otherwise be reachable simply by living inside it.
mgr.add_global_allowed_root(data_dir());
// Enter a plugin context. The deny must hold even inside a globally allowed root.
ScopedPluginAuditContext ctx("test_plugin", "");
// Enter a plugin context. The deny must hold in Loading mode, which every scope runs in.
ScopedPluginAuditContext ctx("test_plugin", "", PluginAuditManager::AuditMode::Loading);
const fs::path conf = fs::path(data_dir()) / (SLIC3R_APP_KEY ".conf");
const fs::path token = fs::path(data_dir()) / secret_constants::USER_SECRET_FILENAME;
@@ -103,13 +103,6 @@ TEST_CASE("Plugin audit deny beats allowed roots", "[audit]")
CHECK(decision.allowed);
}
SECTION("a file outside the allowed root is blocked for reads as well as writes")
{
AuditDecision decision = mgr.check_open((fs::path(data_dir()).parent_path() / "outside.txt").string(), "r");
CHECK_FALSE(decision.allowed);
CHECK(decision.reason == "outside allowed root");
}
SECTION("writing the app config is blocked despite data_dir() being allowed")
{
AuditDecision decision = mgr.check_open(conf.string(), "w");
@@ -117,14 +110,16 @@ TEST_CASE("Plugin audit deny beats allowed roots", "[audit]")
CHECK(decision.reason == "denied filename");
}
SECTION("reading the app config is blocked despite the allowed root")
SECTION("reading the app config is blocked even though Loading exempts reads")
{
// Without the deny, a read in Loading mode short-circuits to allow. The deny sits above
// that exemption, so this must still be blocked.
AuditDecision decision = mgr.check_open(conf.string(), "r");
CHECK_FALSE(decision.allowed);
CHECK(decision.reason == "denied filename");
}
SECTION("reading the cloud refresh token is blocked")
SECTION("reading the cloud refresh token is blocked in Loading mode")
{
AuditDecision decision = mgr.check_open(token.string(), "r");
CHECK_FALSE(decision.allowed);
@@ -155,7 +150,7 @@ TEST_CASE("Plugin audit deny beats a plugin's own scoped root", "[audit]")
const fs::path plugin_dir = fs::path(data_dir()) / "plugins" / "test_plugin";
fs::create_directories(plugin_dir);
ScopedPluginAuditContext ctx("test_plugin", "");
ScopedPluginAuditContext ctx("test_plugin", "", PluginAuditManager::AuditMode::Loading);
mgr.add_scoped_allowed_root(plugin_dir);
SECTION("the plugin's own non-denied file opens for read and write")

View File

@@ -112,18 +112,6 @@ TEST_CASE("install-state sidecar is the source of truth for a cloud plugin's ins
REQUIRE(read_install_state(plugin_dir, state));
CHECK(state.installed_version == "1.2.0");
state.permissions.networking = {"api.example.com", "192.168.45.6"};
state.permissions.fs_read = {"/path/to/read"};
state.permissions.fs_write = {"/path/to/write"};
REQUIRE(write_install_state(plugin_dir, state));
// Permission data is persisted in the same sidecar as the installation metadata.
PluginInstallState persisted;
REQUIRE(read_install_state(plugin_dir, persisted));
CHECK(persisted.permissions.networking == state.permissions.networking);
CHECK(persisted.permissions.fs_read == state.permissions.fs_read);
CHECK(persisted.permissions.fs_write == state.permissions.fs_write);
// Reading the sidecar back onto a freshly-scanned descriptor (whose header version is still
// 1.0.0) must surface the cloud-installed 1.2.0. This is what lets update_cloud_metadata compare
// the cloud's latest version against the installed version instead of the stale header, so an
@@ -132,4 +120,4 @@ TEST_CASE("install-state sidecar is the source of truth for a cloud plugin's ins
scanned.version = "1.0.0"; // as parsed from the unchanged PEP723 header
read_install_state(plugin_dir, scanned);
CHECK(scanned.installed_version == "1.2.0");
}
}