D2L Brightspace DataHub SQL Schema

This repository contains SQL DDL (Data Definition Language) scripts to create local database tables that mirror the D2L Brightspace DataHub (BDS) Data Sets. These scripts are designed to facilitate ETL processes, allowing institutions to store, query, and analyze Brightspace data in a relational environment.

📌 Overview

D2L DataHub provides bulk data exports (CSVs) for various LMS activities. This project provides the structured schema required to host that data in a local SQL database (PostgreSQL, MySQL, or SQL Server), ensuring data integrity and optimized query performance.

📂 Repository Structure

The scripts are organized by functional domain to match the Brightspace Data Sets:

users/: Schema for user profiles, logins, and roles.
org_structure/: Tables for Org Units, Course Offerings, Templates, and Semesters.
enrollments/: Tracking user enrollments across course sections.
grades/: Grade objects, results, schemes, and category details.
content/: Course content structure and user progress tracking.
quizzes/: Comprehensive quiz data, including attempts and question-level responses.
assignments/: Assignment folders, submissions, and feedback.

🚀 Getting Started

Prerequisites

A running SQL database instance (MySQL 8.0+, PostgreSQL 13+, or SQL Server 2019+).
Access to D2L Brightspace DataHub to download the .csv (or .zip) data sets.

Installation

Clone the repository:

git clone https://github.com/your-repo/d2l-datahub-sql.git
cd d2l-datahub-sql

Execute the Schema Scripts: You can run the full setup or individual files depending on your needs.
```
-- Example for PostgreSQL
\i core/create_all_tables.sql
```

🛠 ETL Workflow Recommendation

To keep your local data warehouse synchronized with Brightspace:

Extract: Download the latest Differential or Full Data Sets from DataHub.
Stage: Load the raw CSV data into a staging schema (e.g., stg_bds).
Transform/Load: Use an UPSERT logic to move data from staging into the tables created by this repository, handling IsDeleted flags and updated timestamps.

📊 Schema Standards

Primary Keys: Defined based on D2L's unique identifiers (e.g., UserId, OrgUnitId).
Data Types: Mapped to handle large scale data (e.g., BIGINT for IDs, TEXT for long strings, and TIMESTAMP for dates).
Indexing: Essential foreign keys are indexed by default to optimize joins across the LMS schema.

🤝 Contributing

If you encounter missing fields from newer DataHub versions or want to contribute optimizations for a specific SQL dialect:

Fork the Project.
Create your Feature Branch (git checkout -b feature/NewDataSet).
Commit your Changes (git commit -m 'Add UserAttribute data set').
Push to the Branch (git push origin feature/NewDataSet).
Open a Pull Request.

Disclaimer: This project is not officially affiliated with D2L Corporation. Please refer to the Brightspace DataHub Documentation for the most up-to-date field specifications.