Learn the benefits and drawbacks of using MDS staging tables or web services to load and update your master data. Find out which method suits your performance, flexibility, and security needs.

Staging tables are like the unsung heroes of data management - they do all the heavy lifting behind the scenes! 🏋️♂️ With the right setup, they can handle massive amounts of data without breaking a sweat, making sure your MDS web app stays as cool as a cucumber. 🥒 But remember, with great power comes great responsibility - you've got to keep those tables in check to avoid any data disasters. So, roll up your sleeves and get ready to manage those staging tables like a pro! 💪 #DataManagement #StagingTables #MDS

Last updated on Sep 7, 2024

What are the benefits and drawbacks of using MDS staging tables vs. web services?

Master Data Services (MDS) is a tool that helps you manage and maintain the quality of your master data across different systems and applications. MDS provides two ways to load and update your data: staging tables and web services. Both methods have their advantages and disadvantages, depending on your requirements and preferences. In this article, we will compare and contrast the benefits and drawbacks of using MDS staging tables vs. web services.

1 Staging tables

Staging tables are temporary tables that store the data you want to load or update in MDS. With SQL Server Integration Services (SSIS) or other tools, you can populate the staging tables with data from various sources and use the MDS staging process to validate and apply the changes to the MDS database. Staging tables provide high performance and scalability, allowing you to load and update large volumes of data in batches without affecting the MDS web application or web service. They also offer flexibility and control, allowing you to customize and optimize the data transformation and validation logic before sending it to MDS. Furthermore, staging tables enable error handling and recovery, allowing you to review and correct errors before or after the MDS staging process. However, they also come with drawbacks such as complexity and maintenance, security and access issues, as well as synchronization and consistency challenges. Thus, it is important to create and manage staging tables properly, grant appropriate permissions and roles to users or applications that access them, protect sensitive data from unauthorized or accidental changes, coordinate and schedule data loading/updating between staging tables/MDS database, and handle potential conflicts/discrepancies that may arise from concurrent/delayed data operations.

Add your perspective

JL Suarez, MBA

🏆 50+ LinkedIn Top Voice: 🚀Manager of Enterprise Data Services at Holiday Inn Club Vacations🌐: Inspiring Innovation & Leadership in Business Intelligence📊
Report contribution
Staging tables are like the unsung heroes of data management - they do all the heavy lifting behind the scenes! 🏋️♂️ With the right setup, they can handle massive amounts of data without breaking a sweat, making sure your MDS web app stays as cool as a cucumber. 🥒 But remember, with great power comes great responsibility - you've got to keep those tables in check to avoid any data disasters. So, roll up your sleeves and get ready to manage those staging tables like a pro! 💪 #DataManagement #StagingTables #MDS

Like

2 Web services

Web services are application programming interfaces (APIs) that allow you to interact with MDS programmatically by sending XML requests and receiving XML responses. Benefits of web services include simplicity, convenience, security, access, synchronization, and consistency. However, web services also have drawbacks such as low performance and scalability, limited flexibility and control, and difficulties with error handling and recovery. You may experience slow response times or timeouts when loading or updating large amounts of data. You also have to follow the predefined schema and format of the XML requests and responses, and rely on the MDS business rules and validations. Additionally, you must handle and correct errors in the XML requests and responses, as well as use the MDS web service operations to track and monitor the status of your data operations.

Add your perspective

3 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?

Add your perspective

Jeff Dean

MDM Professional
Report contribution
Even though the data structure could be identical, analysis should be done on the performance implications of a one-time conversion versus on-going operations. While conversion could entail millions or even hundreds of millions of records, daily operations may only involve hundreds of updates. In general calling web services millions of times involves substantial overhead even if the call/response is only a tenth of a second it would take nearly 6 days to load 5 million records (multi-threading can help if contention can be avoided). In daily operations (even in a UX interaction) sub second response is fine, but this can be untenable for conversion time lines. In general, staging tables are a better option for massive volumes of data.

Like

What are the benefits and drawbacks of using MDS staging tables vs. web services?

1

2

3

1 Staging tables

2 Web services

3 Here’s what else to consider

Master Data Management

Rate this article

Thanks for your feedback

More articles on Master Data Management

More relevant reading