Check and repair byte-escaped text in a BioticExplorer database

Databases compiled or updated by BioticExplorerServer <= 0.8.8 under a non-UTF-8 locale (for example a background Rscript started with LANG unset) store Norwegian characters as literal escape text, so that "blåkveite" becomes "bl<c3><a5>kveite". This affects every text column: species, platform and mission-type names, and all comment fields. The function finds such values in all tables of the database, decodes them back to UTF-8 in place and re-indexes the database. Use dryRun = TRUE to only check. An alternative to repairing is to re-download the whole database with compileDatabase(overwrite = TRUE) using BioticExplorerServer >= 0.8.9.

Usage

repairDatabaseEncoding(
  dbPath = defaultDbPath(),
  dbIndexFile = file.path(dbPath, "dbIndex.rda"),
  dbName = NULL,
  dryRun = FALSE
)

Arguments

dbPath

Character string specifying the folder where the duckdb and dbIndex files should be saved. Defaults to defaultDbPath(), which is ~/IMR_biotic_BES_database on macOS and Linux and %USERPROFILE%\IMR_biotic_BES_database on Windows, where R's ~ expansion may otherwise land the database in a OneDrive-synchronized Documents folder.

dbIndexFile

Character string giving the path to the index file rebuilt after a repair. NULL skips re-indexing.

dbName

Character string or NULL. If NULL uses the default name ("bioticexplorer").

dryRun

Logical. If TRUE, opens the database read-only and only reports the affected tables and columns.

Value

A data frame with one row per affected table and column, giving the number of distinct values and rows. Returned invisibly unless dryRun = TRUE. A zero-row data frame means the database has no byte-escaped text.

Details

Close all other connections to the database (R sessions, BioticExplorer) before repairing. Only escape sequences for bytes >= 0x80 that decode to valid UTF-8 are changed, so legitimate text such as "<10>" is left alone.

Author

Mikko Vihtakari

Examples

if (FALSE) { # \dontrun{
repairDatabaseEncoding(dryRun = TRUE) # check
repairDatabaseEncoding()              # repair and re-index
} # }