Simple Solution. Brighter Futures. TSDS Technical Course 4:

Simple Solution. Brighter Futures. TSDS Technical Course 4:

Simple Solution. Brighter Futures. TSDS Technical Course 4: Managing Data Loads into the PEIMS Data Mart Document Number TSDS-Tech-L004-D003 Click to Agenda Course edit Master title style 2 Review the TSDS High Level End User Process Map

Learn how to schedule data promotions from the ODS to the PEIMS Data Mart (PDM) Verify that the data promotion has successfully completed Understand how to troubleshoot and resolve issues that may occur during the data promotion Course Click toObjectives edit Master title style 3

The participant will be able to: Explain the TSDS High Level End User Process Map Access and navigate PDM Schedule a data promotion from the ODS to the PDM

Verify the results of the data promotion Troubleshoot the data promotion process Click to Prerequisites edit Master title style Course 4 Prerequisites that are needed prior to this training:

Training Prerequisites: Participants should attend the TSDS Technical Course 1: Overview of TSDS and TSDS High Level End User Process (or view the online version posted on Project Share) Participants should attend the TSDS Technical Course 2: TSDS Client-Side Validation Tool Participants should complete TSDS Technical Course 3: Loading Data into the ODS

Technical Prerequisites: Participants will need a TEAL ID to access the TSDS Portal and have permission to access ODS Participants will need a Project Share user ID Participants should have a working knowledge of TEDS ClickTerms Key

to edit Master title style 5 Term Definition EDW (Education Data Warehouse) This is the single data repository that feeds the PEIMS and the studentGPS collections DTU (Data Transfer Utility) The DTU is an FTP client that transfers files stored at

the LEA to the eData Manager (eDM) eDM (eData Manager) The portal through which LEAs can manually submit data and monitor data submissions ETL ETL means Extract, Transform, Load. This refers to the process of moving data from one system to another (like SIS to ODS) and transforming the data to meet the requirements of the destination environment ODS (Operational Data Store)

This is the actual data warehouse in the TSDS system PDM (PEIMS Data Mart) The PDM is the data mart that pulls data from the ODS and directly feeds the PEIMS application ClickTerms Key to edit(contd) Master title style 6 Term

Definition DDM (Dashboard Data Mart) The DDM is the data mart that pulls data from the ODS and directly updates the studentGPS Dashboards XML Interchange File TEA uses XML Interchange Files as the vehicle to transfer data Data Promotion The process where data is extracted from the ODS and

is transferred do the PEIMS Data Mart TSDS High Level End User Process Map Overview 7 Click to editLevel Master title style TSDS High End User Process Map

8 Click here to view the TSDS High Level End User Process Map www.projectsharetx.org Click Promotion to edit Master title style Data to the PDM 9 Review: Loading Data into the ODS

10 TSDS eDM: TSDS High Level End User Click toMap edit Master title style Process 11 Click to editFunctions Master title style TSDS eDM: 12

The TSDS eData Manager (TSDS eDM): TSDS eDM automatically picks up files that have been successfully transferred by the TSDS DTU TSDS eDM runs the XML Interchange files through the data validations and updates the Operational Data Store Users can also manually upload XML Interchange Files through TSDS eDM

Error files are generated at initial file validation and again during the ETL process The user will be notified about errors through system messages received through the TSDS portal message center The user can also verify that the ODS has been updated through TSDS eDM TSDS eDM: Received XML Interchange Click to edit Master title style

Files 13 This section specifically discusses the process once the file has been received by TSDS eDM The files can be transferred by the TSDS DTU

The files can be manually submitted Click to editTSDS Master titleData styleFlow TSDS eDM: eDM 14 TSDS eDM picks up XML Interchange file or user manually

submits the file The data passes through file validation If the file fails validation the user needs to resubmit the source system file

If the file passes validation, then it is passed to Batch Manager The ETL process kicks off The ODS is updated Error files may be generated PEIMS Data Mart (PDM) Overview 15

Click to editofMaster title style Process Overview Data Loading 16 LEA Source Systems Interchange extracts are created from source systems in XML format Must align with TEDS Client Side Data Transfer

eDM (Manage Data PEIMS Dashboards Data Validation Tool Utility (DTU) Loads) Application Mart Downloaded from

TSDS Portal Used by LEAs & ESCs to validate XML interchanges prior to loading Errors in interchanges must be corrected in source systems & revalidated A secure FTP file transfer client utility Downloaded from TSDS Portal & installed at LEA/ESC Facilitates On Demand & Scheduled

file transfers Sends XML files to eDM Uses Service Account credentials for login & password User selects files to be transferred File name is validated User can view transfer status Automatically picks up files that have been transferred by the DTU Runs files though

data validations & updates the ODS Users can also manually upload files Error files are generated at initial file validation & again during ETL process Errors in interchanges must be corrected in source systems & revalidated User can verify that the ODS has been updated by viewing status in the eDM

User schedules and can monitor the promotion of data from the ODS to the PDM User specifies collection and submission (e.g. Fall, First) User selects Category (e.g. Staff) and Sub Category(ies) to be loaded (e.g. Employment Payroll Summary) User can monitor the status

Data is loaded from ODS to a staging DDM If load successful, then it is moved to the production DDM for studentGPS display Metric calculations occur during the load of data from the ODS to the DDM Data is automatically refreshed nightly, no user interaction needed System will prevent

incomplete data from being loaded Click to edit Master title style Overview 17 The PEIMS Application allows the end users to promote data from the Operation Data Store (ODS) to the PEIMS Data Mart (PDM) Users can manage the quality of the TSDS PEIMS Data Submission through Validations and the QA Reports

Users can manage and monitor the LEA or ESC acceptance and approval process Users can request an extension from TEA Click to edit Promote Data Master Load title style Loading data from the ODS to the PDM

Logging in to the TSDS Portal 19 Click to edit Logging In Master title style 20 Log in via the TSDS Portal using your TEAL username and password Select the Texas Student Data System Portal from the list of applications PDM Navigation

21 Click to editPDM Accessing Master title style 22 Click to edit Promote Loaded Master Data title Menu styleLink 23

PEIMS: Schedule Data Promotion 24 Click to edit Schedule Data Master Promotion title style Selections 25 Collection School Year

Submission Categories Subcategories Load Date & Time Job Name Click to Data Submit edit Master Promotion title Schedule style 26

User enters parameters and then selects either Reset or Next Click to Data Submit edit Master Promotion title Schedule style 27 Review data promotion and

the click Submit Click to edit Master Confirmation Message title style 28 Confirmation message Monitor Data Promotion 29 Click to edit

Monitor DataMaster Promotions title style Menu Link 30 Click to edit Monitor DataMaster Promotions title style 31 Status Data Promotion Info

Cancel Click toData Search edit Master Promotion title style 32 Search parameters Sort columns using arrows ClickDetails

Job to edit Master title style 33 Select the Job Name ClickDetails: Job to edit Failed Master title style 34 Guided Practice: Data Promotions 35

Click to Practice Guided edit Master title style 36 From this point forward, you will be completing a narrated simulation of the PEIMS Application, specifically scheduling and monitoring data promotions. You will interact with the simulation just as if you were clicking directly into the application. The simulation will provide narrated guidance along the way

Lets log in to Project Share to begin Click Here to Begin Troubleshooting 37 Click to edit Master title style Troubleshooting 38 Data promotions from the ODS to the PDM will occasionally result in a failed status from either a system level issue or a data issue

The LEA Data Steward should: Check the failed error log for an incomplete data submission If that does not identify and resolve the issue, then open a support ticket in TIMS Click to Error Files edit Master title style 39

Placeholder: Error Files Click to editPlan Escalation Master title style 40 All system level or data issues require that a support ticket is opened in TIMS Once the support ticket has been opened in TIMS, the request is routed to the appropriate

resource to resolve the issue Guided Practice: Troubleshooting 41 Click to Practice edit Master title style Guided 42 The Troubleshooting Exercise is located on Project Share

We will collaborate with our colleagues to articulate the action plan to resolve issues that may occur during the data promotion Lets log in to Project Share to begin Click Here to Begin Wrap Up, Knowledge Check, Survey, and Questions 43 Click to Wrap Upedit Master title style

44 Today we talked about: TSDS High Level End User Process Map The basics of loading data into the ODS Scheduling data promotions from the ODS to the PEIMS Data Mart (PDM)

Verifying that the data promotion has successfully completed Troubleshooting issues that may occur during the data promotion What did you learn? How does data promotion to the PDM fit into the TSDS End User Process? What are the basic components of loading data into the ODS?

How does an end user schedule a data promotion to the PDM? Click to editCheck Master title style Knowledge 45 Please click the Knowledge Check link in Project Share and take ten minutes to answer questions about this training session. www.projectsharetx.org Click to edit Master title style Training

Survey 46 Please click the survey link in Project Share and take five minutes to answer questions about this training session. www.projectsharetx.org 47 Questions? www.TexasStudentDataSystem. org

Recently Viewed Presentations