Logs: scrub every older folder in one start, a few at a time

On device, 27 of 31 migrated folders still had .scrubbed-2 a minute after
start: the pass was still running, ~30 s per folder (some 30 MB of Steam
logs each), one folder after another. The start pass now runs folders in
parallel (half the cores, at most 4, background priority) and logs
"logs: scrubbed N folders under r3". Migration strips the moved folders'
markers so that same pass takes them; old markers go whether or not a
folder's pass succeeds, and an unfinished folder is taken up next start.

app/src/main/java/com/droiddeck/launcher/session/LogMigration.kt

@@ -9,8 +9,8 @@ import java.io.File
 /**
  * Once, on the first start of a build with private logs: the session folders, tools/ and the loose
  * tool logs that earlier builds left in `Download/DroidDeck/` move into `files/logs/`. Best effort:
- * an item that cannot be moved stays where it is. Moved session folders are scrubbed again under the
- * current rules ([SessionArtifacts.scrubMoved]). Nothing else in that folder is touched - game
+ * an item that cannot be moved stays where it is. Moved session folders lose their old scrub markers
+ * ([SessionArtifacts.unmarkMoved]), so [SessionArtifacts.scrubOlder] scrubs them under the current rules. Nothing else in that folder is touched - game
  * save backups (Saves/) stay public - and after this pass the app never reads it again.
  */
 object LogMigration {
@@ -40,9 +40,10 @@ object LogMigration {
                 if (SessionPaths.isSessionFolder(dest)) movedFolders.add(dest)
             } else Log.w(TAG, "could not move ${f.name}; left in place")
         }
-        // Earlier builds scrubbed these under older rules (or not at all) and marked them done;
-        // they go through the current redactor before anything can share them.
-        SessionArtifacts.scrubMoved(context, movedFolders)
+        // Earlier builds scrubbed these under older rules (or not at all) and marked them done:
+        // without their markers, the pass over older folders that follows at the same start puts
+        // them through the current redactor.
+        SessionArtifacts.unmarkMoved(movedFolders)
         try { marker.writeText("moved $moved item(s)\n") } catch (e: Exception) { Log.w(TAG, "could not mark the move: ${e.message}") }
         if (moved > 0) Log.i(TAG, "moved $moved log item(s) from $legacy into $target")
     }

app/src/main/java/com/droiddeck/launcher/session/SessionArtifacts.kt

@@ -71,32 +71,48 @@ object SessionArtifacts {
     }
 
     /**
-     * For session folders not yet scrubbed under the redactor's current rules - written before
+     * Every session folder not yet scrubbed under the redactor's current rules - written before
      * every file was scrubbed, before the device's own addresses or the Steam account were, or
-     * moved in from Download/ ([LogMigration]): the whole folder, steam/ and every other subfolder
-     * included, through the redactor. A marker records it. Runs at app start with
-     * [finishAbandoned].
+     * moved in from Download/ ([LogMigration] strips their markers first, [unmarkMoved]): the
+     * whole folder, steam/ and every other subfolder included, through the redactor, and the
+     * current marker written in place of the old ones. Runs at app start with [finishAbandoned].
+     *
+     * A folder holds some 30 MB of Steam logs, about half a minute's work, so folders go through
+     * a few at a time; one the process does not get to keeps its old marker and is taken up at
+     * the next start.
      */
     @Synchronized
     fun scrubOlder(context: Context) {
         val current = SessionPaths.current()
         val dirs = LinuxRuntime.logDir(context).listFiles { f ->
             SessionPaths.isSessionFolder(f) && f != current && !File(f, SCRUBBED_MARKER).exists()
-        } ?: return
-        scrubFolders(context, dirs.toList())
-        if (dirs.isNotEmpty()) Log.i(TAG, "scrubbed ${dirs.size} older session folder(s)")
-    }
-
-    /** [dirs] (session folders [LogMigration] just moved in) through the redactor, whatever their markers say. */
-    @Synchronized
-    fun scrubMoved(context: Context, dirs: List<File>) {
-        scrubFolders(context, dirs.filter { SessionPaths.isSessionFolder(it) && it != SessionPaths.current() })
-    }
-
-    private fun scrubFolders(context: Context, dirs: List<File>) {
+        }?.toList() ?: return
         if (dirs.isEmpty()) return
         LogRedactor.learnFromRuntime(LinuxRuntime.rootDir(context))
-        dirs.forEach { scrubAndMark(it) }
+        val workers = (Runtime.getRuntime().availableProcessors() / 2).coerceIn(1, 4)
+        val pool = java.util.concurrent.Executors.newFixedThreadPool(workers) { r ->
+            Thread({ android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_BACKGROUND); r.run() }, "scrub-logs")
+        }
+        val done = try {
+            pool.invokeAll(dirs.map { dir -> java.util.concurrent.Callable { scrubAndMark(dir) } })
+                .count { f -> try { f.get() } catch (e: Exception) { false } }
+        } finally {
+            pool.shutdown()
+        }
+        val failed = dirs.size - done
+        Log.i(TAG, "logs: scrubbed $done folders under r${LogRedactor.RULES_VERSION}" +
+            if (failed > 0) ", $failed not fully (again at the next start)" else "")
+    }
+
+    /**
+     * Folders [LogMigration] just moved in lose every scrub marker an earlier build left, so
+     * [scrubOlder], which runs right after it, puts them through the current rules whatever those
+     * markers claimed.
+     */
+    fun unmarkMoved(dirs: List<File>) {
+        dirs.filter { SessionPaths.isSessionFolder(it) }.forEach { dir ->
+            dir.listFiles { f -> OLD_SCRUBBED_MARKER.matches(f.name) || f.name == SCRUBBED_TREE_MARKER }?.forEach { it.delete() }
+        }
     }
 
     /**
@@ -155,16 +171,17 @@ object SessionArtifacts {
      * Records [record]'s folder as scrubbed - every file went through without a failure - or
      * leaves it unmarked, so the next share and the pass over older folders scrub it again.
      */
-    private fun markScrubbed(record: Record) {
+    private fun markScrubbed(record: Record): Boolean {
         val dir = record.dir
         val tree = File(dir, SCRUBBED_TREE_MARKER)
+        // A marker from older rules vouches for nothing now, whether or not this pass succeeded.
+        dir.listFiles { f -> OLD_SCRUBBED_MARKER.matches(f.name) }?.forEach { it.delete() }
         if (record.failed) {
             tree.delete()
             Log.w(TAG, "$dir is not fully scrubbed; it is scrubbed again when shared")
-            return
+            return false
         }
         try {
-            dir.listFiles { f -> OLD_SCRUBBED_MARKER.matches(f.name) }?.forEach { it.delete() }
             File(dir, SCRUBBED_MARKER).writeText("scrubbed ${now()}\n")
             val lines = StringBuilder("rules ${LogRedactor.RULES_VERSION}\n")
             record.files.forEach { (f, s) ->
@@ -172,8 +189,11 @@ object SessionArtifacts {
                     .append(f.relativeTo(dir).path).append('\n')
             }
             tree.writeText(lines.toString())
+            true
         } catch (e: Exception) {
             tree.delete()
+            File(dir, SCRUBBED_MARKER).delete()
+            false
         }
     }
 
@@ -195,7 +215,7 @@ object SessionArtifacts {
     }
 
     /** The end-of-session scrub of [dir] on its own (steam/ already in place); also the pass over older and moved folders. */
-    internal fun scrubAndMark(dir: File) = markScrubbed(scrubTree(Record(dir)))
+    internal fun scrubAndMark(dir: File): Boolean = markScrubbed(scrubTree(Record(dir)))
 
     /**
      * [record]'s folder and every folder under it (steam/, droiddeck-esync/): the session script

app/src/test/java/com/droiddeck/launcher/session/SessionLogShareZipTest.kt

@@ -53,39 +53,55 @@ class SessionLogShareZipTest {
         assertEquals(line, com.droiddeck.launcher.core.LogRedactor.redactForShare(line))
     }
 
-    // A folder an earlier build scrubbed and marked under older rules, its copied Steam UI log
-    // still naming the account: the pass at app start (and after the move out of Download/)
-    // scrubs it again on disk.
-    private fun olderFolder(name: String): File {
-        val dir = File(LinuxRuntime.logDir(app), name).apply { mkdirs() }
-        File(dir, ".scrubbed-2").writeText("scrubbed earlier\n")
+    // Folders an earlier build scrubbed and marked under older rules, their copied Steam UI log
+    // still naming the account: one app start (the move out of Download/, then the pass over
+    // older folders) leaves every one scrubbed on disk under the current rules.
+    private fun olderFolder(parent: File, name: String, marker: String = ".scrubbed-2"): File {
+        val dir = File(parent, name).apply { mkdirs() }
+        File(dir, marker).writeText("scrubbed earlier\n")
+        File(dir, SessionArtifacts.SCRUBBED_TREE_MARKER).writeText("rules 2\n")
         File(dir, "steam").apply { mkdirs() }.resolve("webhelper_js.txt").writeText(
             "[2026-10-01 20:00:00] SteamUI: INFO: Login: OnLoginStateChange someone.masked@example.com 2 1 0 0\n" +
                 "[2026-10-01 20:00:01] SteamUI: INFO: Login: OnLoginStateChange maskeduser42 2 1 0 0\n" +
-                "[2026-10-01 20:00:02] SteamUI: INFO: Login: OnLoginStateChange  0 1 0 0\n"
+                "[2026-10-01 20:00:02] SteamUI: INFO: Login: OnLoginStateChange  0 1 0 0\n" +
+                "0024:trace:seh:dispatch_exception code=c0000005 flags=0\n"
         )
         return dir
     }
 
     private fun assertScrubbed(dir: File) {
         val text = File(dir, "steam/webhelper_js.txt").readText()
-        for (secret in listOf("someone.masked", "example.com", "maskeduser42")) assertFalse("$secret survived", text.contains(secret))
+        for (secret in listOf("someone.masked", "example.com", "maskeduser42")) assertFalse("$secret survived in $dir", text.contains(secret))
         assertTrue(text.contains("Login: OnLoginStateChange <redacted:account> 2 1 0 0"))
         assertTrue(text.contains("Login: OnLoginStateChange  0 1 0 0"))
-        assertFalse(File(dir, ".scrubbed-2").exists())
-        assertTrue(File(dir, ".scrubbed-r${com.droiddeck.launcher.core.LogRedactor.RULES_VERSION}").exists())
+        assertTrue(text.contains("code=c0000005"))
+        val markers = dir.list()!!.filter { it.startsWith(".scrubbed-") && it != SessionArtifacts.SCRUBBED_TREE_MARKER }
+        assertEquals("$dir", listOf(".scrubbed-r${com.droiddeck.launcher.core.LogRedactor.RULES_VERSION}"), markers)
+        assertTrue(File(dir, SessionArtifacts.SCRUBBED_TREE_MARKER).readText()
+            .startsWith("rules ${com.droiddeck.launcher.core.LogRedactor.RULES_VERSION}\n"))
     }
 
     @Test fun aFolderMarkedUnderOlderRulesIsScrubbedAgainAtStart() {
-        val dir = olderFolder("2026-10-01-01-steam")
+        val dir = olderFolder(LinuxRuntime.logDir(app), "2026-10-01-01-steam")
         SessionArtifacts.scrubOlder(app)
         assertScrubbed(dir)
     }
 
-    @Test fun aMovedFolderIsScrubbedWhateverItsMarkersSay() {
-        val dir = olderFolder("2026-10-01-02-steam")
-        File(dir, ".scrubbed-r${com.droiddeck.launcher.core.LogRedactor.RULES_VERSION}").writeText("claimed\n")
-        SessionArtifacts.scrubMoved(app, listOf(dir))
-        assertScrubbed(dir)
+    @Test fun everyMigratedFolderIsScrubbedUnderTheCurrentRulesInOneStart() {
+        val legacy = LinuxRuntime.legacyLogDir().apply { mkdirs() }
+        val names = (1..9).map { "2026-10-0${it}-01-steam" }
+        names.forEachIndexed { i, name ->
+            // Old markers of every kind an earlier build left, one even claiming the current rules.
+            olderFolder(legacy, name, marker = listOf(".scrubbed-2", ".scrubbed-1", ".scrubbed-r${com.droiddeck.launcher.core.LogRedactor.RULES_VERSION}")[i % 3])
+        }
+        File(legacy, "Saves").apply { mkdirs() }.resolve("keep.txt").writeText("save\n")
+
+        // What MainActivity runs at start.
+        LogMigration.run(app)
+        SessionArtifacts.scrubOlder(app)
+
+        val logs = LinuxRuntime.logDir(app)
+        for (name in names) assertScrubbed(File(logs, name))
+        assertEquals(listOf("Saves"), legacy.list()!!.toList())
     }
 }

docs/progress-log.md

@@ -41,8 +41,8 @@ the timeline, then lessons and backlog. Companion to the README (what the app *d
   those folders carried a scrub marker from before the account rules, and the marker never
   expired. The marker is now named after the redactor's rules version (`.scrubbed-r3`), so every
   folder scrubbed under older rules goes through again at app start, steam/ and every subfolder
-  included; folders moved out of `Download/DroidDeck` are scrubbed after the move whatever their
-  markers say (`SessionArtifacts.scrubMoved`). New rules: `OnLoginStateChange <anything>`, and
+  included; folders moved out of `Download/DroidDeck` lose every old marker
+  (`SessionArtifacts.unmarkMoved`), so the same pass takes them whatever those markers said. New rules: `OnLoginStateChange <anything>`, and
   `AccountName` / `account_name` / `username` / `login` fields (`login` only as `login=`, a quoted
   key or a VDF pair, so the "Login:" label stays). The `code=` rule no longer touches Proton's
   exception codes (`code=c0000005`, `406d1388`, `80000003`): a bare `code=` is blanked only right
@@ -50,6 +50,14 @@ the timeline, then lessons and backlog. Companion to the README (what the app *d
   never when it is 8 hex digits; exchange / authorization codes are always blanked. Tests:
   `LogRedactorTest` (the login line with an email and with a plain name), `SecretScrubTest`,
   `SessionLogShareZipTest` (older and moved folders scrubbed on disk).
+- **Device check (31 migrated folders):** only 4 carried `.scrubbed-r3` a minute in. Nothing was
+  skipped - the pass was still running: one folder holds ~30 MB of Steam logs, ~30 s through the
+  redactor, one folder after another (about 15 minutes for 31). The start pass now scrubs a few
+  folders at a time (half the cores, at most 4, background priority), old markers go whether or
+  not a folder's pass succeeds, a folder the process does not finish is taken up at the next start,
+  and it logs one line: `logs: scrubbed N folders under r3`. Test: nine migrated folders carrying
+  `.scrubbed-2`, `.scrubbed-1` or a stale `.scrubbed-r3` all end up scrubbed under r3 after one
+  migration + start pass.
 ## 2026-10-08 - `feat/stores`: GOG, Epic Games and Amazon Games in the launcher (in progress)
 
 A new **Stores** section - the three storefronts' libraries and public catalogs, one download