This page walks through creating the Mapped Data
Layer step-by-step using the standard mappings shipped with the
gsm.mapping package. The Mapped Data Layer filters,
aggregates, and joins raw/source data into the standardized datasets
that downstream gsm packages (e.g. gsm.kri)
expect.
Expand the “Setup” section below to see the libraries and source data used, or scroll down to follow the workflow.
# Load libraries
library(gsm.core)
library(gsm.mapping)
library(dplyr)
#>
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#>
#> filter, lag
#> The following objects are masked from 'package:base':
#>
#> intersect, setdiff, setequal, union
# The 12 standard domains used to build a KRI report
core_mappings <- c(
"AE", "COUNTRY", "DATACHG", "DATAENT", "ENROLL", "LB",
"PD", "QUERY", "STUDY", "STUDCOMP", "SDRGCOMP", "SITE", "SUBJ"
)
# Sample raw/source data provided by gsm.core
lSource <- gsm.core::lSourceThe Mapped Data Layer is created in a short, four-step workflow: build the list of mapping workflows, combine their specifications, ingest the raw data, and run the workflows to produce the mapped data.
workr::MakeWorkflowList() reads the YAML mapping
specifications shipped in inst/workflow/1_mappings for each
requested domain.
mappings_wf <- workr::MakeWorkflowList(
strNames = core_mappings,
strPath = "workflow/1_mappings",
strPackage = "gsm.mapping"
)CombineSpecs() collapses the individual domain
specifications into a single specification describing every raw dataset
and column required.
mappings_spec <- CombineSpecs(mappings_wf)Ingest() uses the combined specification to select and
standardize the required columns from the raw/source data.
lRaw <- Ingest(lSource, mappings_spec)
#> ℹ Ingesting data for AE.
#> [INFO] Creating a new temporary DuckDB connection.
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 3000 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> ℹ Ingesting data for ENROLL.
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 1000 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> ℹ Ingesting data for LB.
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 405720 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> ℹ Ingesting data for PD.
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 3000 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> ℹ Ingesting data for SDRGCOMP.
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 100 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> ℹ Ingesting data for STUDCOMP.
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 100 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> ℹ Ingesting data for SUBJ.
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 1000 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> ℹ Ingesting data for DATACHG.
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 435968 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> ℹ Ingesting data for SUBJ.
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 1000 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> ℹ Ingesting data for DATAENT.
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 108992 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> ℹ Ingesting data for QUERY.
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 27248 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> ℹ Ingesting data for SITE.
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 150 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> ℹ Ingesting data for STUDY.
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 1 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.workr::RunWorkflows() executes each mapping workflow
against the ingested data to filter, aggregate, and join it into the
final Mapped Data Layer.
mapped <- workr::RunWorkflows(mappings_wf, lRaw)
#> [INFO] Running 13 Workflows
#> [INFO] Initializing `Mapped_AE` Workflow
#> [INFO] Checking data against spec
#> [INFO] Workflow Step 1 of 1: `=`
#> [INFO] Evaluating 2 parameter(s) for `=`
#> [INFO] lhs = Mapped_AE: No matching data found. Passing 'Mapped_AE' as a string.
#> [INFO] rhs = Raw_AE: Passing lData$Raw_AE.
#> [INFO] Calling `=`
#> [INFO] 3000x11 data.frame saved as `lData$Mapped_AE`.
#> [INFO] Returning results from final step: 3000x11 data.frame`.
#> [INFO] Completed `Mapped_AE` Workflow
#> [INFO] Initializing `Mapped_ENROLL` Workflow
#> [INFO] Checking data against spec
#> [INFO] Workflow Step 1 of 1: `=`
#> [INFO] Evaluating 2 parameter(s) for `=`
#> [INFO] lhs = Mapped_ENROLL: No matching data found. Passing 'Mapped_ENROLL' as a string.
#> [INFO] rhs = Raw_ENROLL: Passing lData$Raw_ENROLL.
#> [INFO] Calling `=`
#> [INFO] 1000x7 data.frame saved as `lData$Mapped_ENROLL`.
#> [INFO] Returning results from final step: 1000x7 data.frame`.
#> [INFO] Completed `Mapped_ENROLL` Workflow
#> [INFO] Initializing `Mapped_LB` Workflow
#> [INFO] Checking data against spec
#> [INFO] Workflow Step 1 of 1: `=`
#> [INFO] Evaluating 2 parameter(s) for `=`
#> [INFO] lhs = Mapped_LB: No matching data found. Passing 'Mapped_LB' as a string.
#> [INFO] rhs = Raw_LB: Passing lData$Raw_LB.
#> [INFO] Calling `=`
#> [INFO] 405720x4 data.frame saved as `lData$Mapped_LB`.
#> [INFO] Returning results from final step: 405720x4 data.frame`.
#> [INFO] Completed `Mapped_LB` Workflow
#> [INFO] Initializing `Mapped_PD` Workflow
#> [INFO] Checking data against spec
#> [INFO] Workflow Step 1 of 1: `=`
#> [INFO] Evaluating 2 parameter(s) for `=`
#> [INFO] lhs = Mapped_PD: No matching data found. Passing 'Mapped_PD' as a string.
#> [INFO] rhs = Raw_PD: Passing lData$Raw_PD.
#> [INFO] Calling `=`
#> [INFO] 3000x6 data.frame saved as `lData$Mapped_PD`.
#> [INFO] Returning results from final step: 3000x6 data.frame`.
#> [INFO] Completed `Mapped_PD` Workflow
#> [INFO] Initializing `Mapped_SDRGCOMP` Workflow
#> [INFO] Checking data against spec
#> [INFO] Workflow Step 1 of 1: `=`
#> [INFO] Evaluating 2 parameter(s) for `=`
#> [INFO] lhs = Mapped_SDRGCOMP: No matching data found. Passing 'Mapped_SDRGCOMP' as a string.
#> [INFO] rhs = Raw_SDRGCOMP: Passing lData$Raw_SDRGCOMP.
#> [INFO] Calling `=`
#> [INFO] 100x5 data.frame saved as `lData$Mapped_SDRGCOMP`.
#> [INFO] Returning results from final step: 100x5 data.frame`.
#> [INFO] Completed `Mapped_SDRGCOMP` Workflow
#> [INFO] Initializing `Mapped_STUDCOMP` Workflow
#> [INFO] Checking data against spec
#> [INFO] Workflow Step 1 of 1: `=`
#> [INFO] Evaluating 2 parameter(s) for `=`
#> [INFO] lhs = Mapped_STUDCOMP: No matching data found. Passing 'Mapped_STUDCOMP' as a string.
#> [INFO] rhs = Raw_STUDCOMP: Passing lData$Raw_STUDCOMP.
#> [INFO] Calling `=`
#> [INFO] 100x6 data.frame saved as `lData$Mapped_STUDCOMP`.
#> [INFO] Returning results from final step: 100x6 data.frame`.
#> [INFO] Completed `Mapped_STUDCOMP` Workflow
#> [INFO] Initializing `Mapped_SUBJ` Workflow
#> [INFO] Checking data against spec
#> [INFO] Workflow Step 1 of 1: `workr::RunQuery`
#> [INFO] Evaluating 2 parameter(s) for `workr::RunQuery`
#> [INFO] df = Raw_SUBJ: Passing lData$Raw_SUBJ.
#> [INFO] strQuery = SELECT * FROM df WHERE enrollyn == 'Y': No matching data found. Passing 'SELECT * FROM df WHERE enrollyn == 'Y'' as a string.
#> [INFO] Calling `workr::RunQuery`
#> [INFO] Creating a new temporary DuckDB connection.
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 762 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> [INFO] 762x14 data.frame saved as `lData$Mapped_SUBJ`.
#>
#> [INFO] Returning results from final step: 762x14 data.frame`.
#>
#> [INFO] Completed `Mapped_SUBJ` Workflow
#>
#> [INFO] Initializing `Mapped_DATACHG` Workflow
#>
#> [INFO] Checking data against spec
#>
#> [INFO] Workflow Step 1 of 2: `dplyr::select`
#>
#> [INFO] Evaluating 4 parameter(s) for `dplyr::select`
#>
#> [INFO] .data = Mapped_SUBJ: Passing lData$Mapped_SUBJ.
#>
#> [INFO] subjid = subjid: No matching data found. Passing 'subjid' as a string.
#>
#> [INFO] subject_nsv = subject_nsv: No matching data found. Passing 'subject_nsv' as a string.
#>
#> [INFO] studyid = studyid: No matching data found. Passing 'studyid' as a string.
#>
#> [INFO] Calling `dplyr::select`
#>
#> [INFO] 762x3 data.frame saved as `lData$Temp_SubjectLookup`.
#>
#> [INFO] Workflow Step 2 of 2: `dplyr::left_join`
#>
#> [INFO] Evaluating 3 parameter(s) for `dplyr::left_join`
#>
#> [INFO] x = Raw_DATACHG: Passing lData$Raw_DATACHG.
#>
#> [INFO] y = Temp_SubjectLookup: Passing lData$Temp_SubjectLookup.
#>
#> [INFO] by is of length 2: Parameter is a vector. Passing as is.
#>
#> [INFO] Calling `dplyr::left_join`
#>
#> [INFO] 435968x5 data.frame saved as `lData$Mapped_DATACHG`.
#>
#> [INFO] Returning results from final step: 435968x5 data.frame`.
#>
#> [INFO] Completed `Mapped_DATACHG` Workflow
#>
#> [INFO] Initializing `Mapped_DATAENT` Workflow
#>
#> [INFO] Checking data against spec
#>
#> [INFO] Workflow Step 1 of 2: `dplyr::select`
#>
#> [INFO] Evaluating 4 parameter(s) for `dplyr::select`
#>
#> [INFO] .data = Mapped_SUBJ: Passing lData$Mapped_SUBJ.
#>
#> [INFO] subjid = subjid: No matching data found. Passing 'subjid' as a string.
#>
#> [INFO] subject_nsv = subject_nsv: No matching data found. Passing 'subject_nsv' as a string.
#>
#> [INFO] studyid = studyid: No matching data found. Passing 'studyid' as a string.
#>
#> [INFO] Calling `dplyr::select`
#>
#> [INFO] 762x3 data.frame saved as `lData$Temp_SubjectLookup`.
#>
#> [INFO] Workflow Step 2 of 2: `dplyr::left_join`
#>
#> [INFO] Evaluating 3 parameter(s) for `dplyr::left_join`
#>
#> [INFO] x = Raw_DATAENT: Passing lData$Raw_DATAENT.
#>
#> [INFO] y = Temp_SubjectLookup: Passing lData$Temp_SubjectLookup.
#>
#> [INFO] by is of length 2: Parameter is a vector. Passing as is.
#>
#> [INFO] Calling `dplyr::left_join`
#>
#> [INFO] 108992x5 data.frame saved as `lData$Mapped_DATAENT`.
#>
#> [INFO] Returning results from final step: 108992x5 data.frame`.
#>
#> [INFO] Completed `Mapped_DATAENT` Workflow
#>
#> [INFO] Initializing `Mapped_QUERY` Workflow
#>
#> [INFO] Checking data against spec
#>
#> [INFO] Workflow Step 1 of 2: `dplyr::select`
#>
#> [INFO] Evaluating 4 parameter(s) for `dplyr::select`
#>
#> [INFO] .data = Mapped_SUBJ: Passing lData$Mapped_SUBJ.
#>
#> [INFO] studyid = studyid: No matching data found. Passing 'studyid' as a string.
#>
#> [INFO] subjid = subjid: No matching data found. Passing 'subjid' as a string.
#>
#> [INFO] subject_nsv = subject_nsv: No matching data found. Passing 'subject_nsv' as a string.
#>
#> [INFO] Calling `dplyr::select`
#>
#> [INFO] 762x3 data.frame saved as `lData$Temp_SubjectLookup`.
#>
#> [INFO] Workflow Step 2 of 2: `dplyr::left_join`
#>
#> [INFO] Evaluating 3 parameter(s) for `dplyr::left_join`
#>
#> [INFO] x = Raw_QUERY: Passing lData$Raw_QUERY.
#>
#> [INFO] y = Temp_SubjectLookup: Passing lData$Temp_SubjectLookup.
#>
#> [INFO] by is of length 2: Parameter is a vector. Passing as is.
#>
#> [INFO] Calling `dplyr::left_join`
#>
#> [INFO] 27248x6 data.frame saved as `lData$Mapped_QUERY`.
#>
#> [INFO] Returning results from final step: 27248x6 data.frame`.
#>
#> [INFO] Completed `Mapped_QUERY` Workflow
#>
#> [INFO] Initializing `Mapped_COUNTRY` Workflow
#>
#> [INFO] Checking data against spec
#>
#> [INFO] Workflow Step 1 of 2: `workr::RunQuery`
#>
#> [INFO] Evaluating 2 parameter(s) for `workr::RunQuery`
#>
#> [INFO] df = Mapped_SUBJ: Passing lData$Mapped_SUBJ.
#>
#> [INFO] strQuery = SELECT country as GroupID, COUNT(DISTINCT subjid) as ParticipantCount, COUNT(DISTINCT invid) as SiteCount FROM df GROUP BY country: No matching data found. Passing 'SELECT country as GroupID, COUNT(DISTINCT subjid) as ParticipantCount, COUNT(DISTINCT invid) as SiteCount FROM df GROUP BY country' as a string.
#>
#> [INFO] Calling `workr::RunQuery`
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 3 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> [INFO] 3x3 data.frame saved as `lData$Temp_CountryCountsWide`.
#>
#> [INFO] Workflow Step 2 of 2: `gsm.mapping::MakeLongMeta`
#>
#> [INFO] Evaluating 2 parameter(s) for `gsm.mapping::MakeLongMeta`
#>
#> [INFO] data = Temp_CountryCountsWide: Passing lData$Temp_CountryCountsWide.
#>
#> [INFO] strGroupLevel = Country: No matching data found. Passing 'Country' as a string.
#>
#> [INFO] Calling `gsm.mapping::MakeLongMeta`
#>
#> [INFO] 6x4 data.frame saved as `lData$Mapped_COUNTRY`.
#>
#> [INFO] Returning results from final step: 6x4 data.frame`.
#>
#> [INFO] Completed `Mapped_COUNTRY` Workflow
#>
#> [INFO] Initializing `Mapped_SITE` Workflow
#>
#> [INFO] Checking data against spec
#>
#> [INFO] Workflow Step 1 of 5: `workr::RunQuery`
#>
#> [INFO] Evaluating 2 parameter(s) for `workr::RunQuery`
#>
#> [INFO] df = Raw_SITE: Passing lData$Raw_SITE.
#>
#> [INFO] strQuery = SELECT invid as GroupID, * FROM df: No matching data found. Passing 'SELECT invid as GroupID, * FROM df' as a string.
#>
#> [INFO] Calling `workr::RunQuery`
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 150 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> [INFO] 150x10 data.frame saved as `lData$Temp_CTMSSiteWide`.
#>
#> [INFO] Workflow Step 2 of 5: `gsm.mapping::MakeLongMeta`
#>
#> [INFO] Evaluating 2 parameter(s) for `gsm.mapping::MakeLongMeta`
#>
#> [INFO] data = Temp_CTMSSiteWide: Passing lData$Temp_CTMSSiteWide.
#>
#> [INFO] strGroupLevel = Site: No matching data found. Passing 'Site' as a string.
#>
#> [INFO] Calling `gsm.mapping::MakeLongMeta`
#>
#> [INFO] 1350x4 data.frame saved as `lData$Temp_CTMSSite`.
#>
#> [INFO] Workflow Step 3 of 5: `workr::RunQuery`
#>
#> [INFO] Evaluating 2 parameter(s) for `workr::RunQuery`
#>
#> [INFO] df = Mapped_SUBJ: Passing lData$Mapped_SUBJ.
#>
#> [INFO] strQuery = SELECT invid as GroupID, COUNT(DISTINCT subjid) as ParticipantCount, COUNT(DISTINCT invid) as SiteCount FROM df GROUP BY invid: No matching data found. Passing 'SELECT invid as GroupID, COUNT(DISTINCT subjid) as ParticipantCount, COUNT(DISTINCT invid) as SiteCount FROM df GROUP BY invid' as a string.
#>
#> [INFO] Calling `workr::RunQuery`
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 143 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> [INFO] 143x3 data.frame saved as `lData$Temp_SiteCountsWide`.
#>
#> [INFO] Workflow Step 4 of 5: `gsm.mapping::MakeLongMeta`
#>
#> [INFO] Evaluating 2 parameter(s) for `gsm.mapping::MakeLongMeta`
#>
#> [INFO] data = Temp_SiteCountsWide: Passing lData$Temp_SiteCountsWide.
#>
#> [INFO] strGroupLevel = Site: No matching data found. Passing 'Site' as a string.
#>
#> [INFO] Calling `gsm.mapping::MakeLongMeta`
#>
#> [INFO] 286x4 data.frame saved as `lData$Temp_SiteCounts`.
#>
#> [INFO] Workflow Step 5 of 5: `dplyr::bind_rows`
#>
#> [INFO] Evaluating 2 parameter(s) for `dplyr::bind_rows`
#>
#> [INFO] Temp_CTMSSite = Temp_CTMSSite: Passing lData$Temp_CTMSSite.
#>
#> [INFO] Temp_SiteCounts = Temp_SiteCounts: Passing lData$Temp_SiteCounts.
#>
#> [INFO] Calling `dplyr::bind_rows`
#>
#> [INFO] 1636x4 data.frame saved as `lData$Mapped_SITE`.
#>
#> [INFO] Returning results from final step: 1636x4 data.frame`.
#>
#> [INFO] Completed `Mapped_SITE` Workflow
#>
#> [INFO] Initializing `Mapped_STUDY` Workflow
#>
#> [INFO] Checking data against spec
#>
#> [INFO] Workflow Step 1 of 9: `workr::RunQuery`
#>
#> [INFO] Evaluating 2 parameter(s) for `workr::RunQuery`
#>
#> [INFO] df = Raw_STUDY: Passing lData$Raw_STUDY.
#>
#> [INFO] strQuery = SELECT studyid as GroupID, * FROM df: No matching data found. Passing 'SELECT studyid as GroupID, * FROM df' as a string.
#>
#> [INFO] Calling `workr::RunQuery`
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 1 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> [INFO] 1x16 data.frame saved as `lData$Temp_CTMSStudyWide`.
#>
#> [INFO] Workflow Step 2 of 9: `gsm.mapping::MakeLongMeta`
#>
#> [INFO] Evaluating 2 parameter(s) for `gsm.mapping::MakeLongMeta`
#>
#> [INFO] data = Temp_CTMSStudyWide: Passing lData$Temp_CTMSStudyWide.
#>
#> [INFO] strGroupLevel = Study: No matching data found. Passing 'Study' as a string.
#>
#> [INFO] Calling `gsm.mapping::MakeLongMeta`
#>
#> [INFO] 15x4 data.frame saved as `lData$Temp_CTMSStudy`.
#>
#> [INFO] Workflow Step 3 of 9: `workr::RunQuery`
#>
#> [INFO] Evaluating 2 parameter(s) for `workr::RunQuery`
#>
#> [INFO] df = Raw_STUDY: Passing lData$Raw_STUDY.
#>
#> [INFO] strQuery = SELECT studyid as GroupID, num_plan_site as SiteTarget, num_plan_subj as ParticipantTarget FROM df: No matching data found. Passing 'SELECT studyid as GroupID, num_plan_site as SiteTarget, num_plan_subj as ParticipantTarget FROM df' as a string.
#>
#> [INFO] Calling `workr::RunQuery`
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 1 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> [INFO] 1x3 data.frame saved as `lData$Temp_CTMSplanned`.
#>
#> [INFO] Workflow Step 4 of 9: `workr::RunQuery`
#>
#> [INFO] Evaluating 2 parameter(s) for `workr::RunQuery`
#>
#> [INFO] df = Raw_SUBJ: Passing lData$Raw_SUBJ.
#>
#> [INFO] strQuery = SELECT studyid as GroupID, COUNT(DISTINCT subjid) as ParticipantCount, COUNT(DISTINCT invid) as SiteCount FROM df WHERE enrollyn == 'Y' GROUP BY studyid: No matching data found. Passing 'SELECT studyid as GroupID, COUNT(DISTINCT subjid) as ParticipantCount, COUNT(DISTINCT invid) as SiteCount FROM df WHERE enrollyn == 'Y' GROUP BY studyid' as a string.
#>
#> [INFO] Calling `workr::RunQuery`
#>
#> [INFO] Creating a new temporary DuckDB connection.
#>
#> duckdb keeps downloaded extensions and secrets in a temporary directory:
#> ℹ /tmp/RtmpeviKzb/duckdb
#> This is removed when the R session ends.
#> • Extensions are re-downloaded each session.
#> • Secrets are lost.
#> ℹ Run duckdb(shared_home = TRUE) (or create ~/.duckdb) to keep them (suitable for most users).
#> ℹ Run duckdb(shared_home = FALSE) to accept the temporary directory (and silence this message).
#> ℹ See ?duckdb_storage for details and alternatives.
#> [INFO] SQL Query complete: 1 rows returned.
#>
#> [INFO] Disconnected from temporary DuckDB connection.
#>
#> [INFO] 1x3 data.frame saved as `lData$Temp_StudyCountsWide`.
#>
#> [INFO] Workflow Step 5 of 9: `dplyr::left_join`
#>
#> [INFO] Evaluating 3 parameter(s) for `dplyr::left_join`
#>
#> [INFO] x = Temp_CTMSplanned: Passing lData$Temp_CTMSplanned.
#>
#> [INFO] y = Temp_StudyCountsWide: Passing lData$Temp_StudyCountsWide.
#>
#> [INFO] by = GroupID: No matching data found. Passing 'GroupID' as a string.
#>
#> [INFO] Calling `dplyr::left_join`
#>
#> [INFO] 1x5 data.frame saved as `lData$Temp_CountTargetsWide`.
#>
#> [INFO] Workflow Step 6 of 9: `gsm.mapping::CalculatePercentage`
#>
#> [INFO] Evaluating 5 parameter(s) for `gsm.mapping::CalculatePercentage`
#>
#> [INFO] data = Temp_CountTargetsWide: Passing lData$Temp_CountTargetsWide.
#>
#> [INFO] strCurrentCol = SiteCount: No matching data found. Passing 'SiteCount' as a string.
#>
#> [INFO] strTargetCol = SiteTarget: No matching data found. Passing 'SiteTarget' as a string.
#>
#> [INFO] strPercVal = PercentSitesActivated: No matching data found. Passing 'PercentSitesActivated' as a string.
#>
#> [INFO] strPercStrVal = SiteActivation: No matching data found. Passing 'SiteActivation' as a string.
#>
#> [INFO] Calling `gsm.mapping::CalculatePercentage`
#>
#> [INFO] 1x7 data.frame saved as `lData$Temp_CountTargetsWide_addsite`.
#>
#> [INFO] Workflow Step 7 of 9: `gsm.mapping::CalculatePercentage`
#>
#> [INFO] Evaluating 5 parameter(s) for `gsm.mapping::CalculatePercentage`
#>
#> [INFO] data = Temp_CountTargetsWide_addsite: Passing lData$Temp_CountTargetsWide_addsite.
#>
#> [INFO] strCurrentCol = ParticipantCount: No matching data found. Passing 'ParticipantCount' as a string.
#>
#> [INFO] strTargetCol = ParticipantTarget: No matching data found. Passing 'ParticipantTarget' as a string.
#>
#> [INFO] strPercVal = PercentParticipantsEnrolled: No matching data found. Passing 'PercentParticipantsEnrolled' as a string.
#>
#> [INFO] strPercStrVal = ParticipantEnrollment: No matching data found. Passing 'ParticipantEnrollment' as a string.
#>
#> [INFO] Calling `gsm.mapping::CalculatePercentage`
#>
#> [INFO] 1x9 data.frame saved as `lData$Temp_CountTargetsWide_addsitepts`.
#>
#> [INFO] Workflow Step 8 of 9: `gsm.mapping::MakeLongMeta`
#>
#> [INFO] Evaluating 2 parameter(s) for `gsm.mapping::MakeLongMeta`
#>
#> [INFO] data = Temp_CountTargetsWide_addsitepts: Passing lData$Temp_CountTargetsWide_addsitepts.
#>
#> [INFO] strGroupLevel = Study: No matching data found. Passing 'Study' as a string.
#>
#> [INFO] Calling `gsm.mapping::MakeLongMeta`
#>
#> [INFO] 8x4 data.frame saved as `lData$Temp_CountTargetsPercs`.
#>
#> [INFO] Workflow Step 9 of 9: `dplyr::bind_rows`
#>
#> [INFO] Evaluating 2 parameter(s) for `dplyr::bind_rows`
#>
#> [INFO] Temp_CTMSStudy = Temp_CTMSStudy: Passing lData$Temp_CTMSStudy.
#>
#> [INFO] Temp_CountTargetsPercs = Temp_CountTargetsPercs: Passing lData$Temp_CountTargetsPercs.
#>
#> [INFO] Calling `dplyr::bind_rows`
#>
#> [INFO] 23x4 data.frame saved as `lData$Mapped_STUDY`.
#>
#> [INFO] Returning results from final step: 23x4 data.frame`.
#>
#> [INFO] Completed `Mapped_STUDY` Workflowmapped is a named list of standardized datasets, one per
mapped domain, ready for use in downstream gsm reporting
workflows.
names(mapped)
#> [1] "Mapped_AE" "Mapped_ENROLL" "Mapped_LB" "Mapped_PD"
#> [5] "Mapped_SDRGCOMP" "Mapped_STUDCOMP" "Mapped_SUBJ" "Mapped_DATACHG"
#> [9] "Mapped_DATAENT" "Mapped_QUERY" "Mapped_COUNTRY" "Mapped_SITE"
#> [13] "Mapped_STUDY"