Metadata Driven Pipelines – Data Engineering with Fabric
What is a metadata driven pipeline? Wikipedia defines metadata as “data that provides information about other data.” As a developer, we can create a non-parameterized pipeline and/or notebook to solve a business problem. However, if we must solve the s …
PySpark Secrets to use with Fabric
PySpark is a powerful language for data manipulation and it’s full of tricks. Let’s discover some of them. Control the Type of a NULL column If you are creating a pysspark dataframe, but one of the columns contains only null values (None), …
Join Me at the PASS Data Community Summit in November!
On November 4 in Seattle, I’m presenting a new pre-conference workshop! Tuning T-SQL for SQL Server 2019 and 2022 You’ve been working with SQL Server for a few years, and you’re comfortable writing queries and reading execution plans. …
Updating Documents in MongoDB
This article is part of Robert Sheldon’s continuing series on Mongo DB. To see all of the items in the series, click here. In the previous articles this series, I demonstrated various ways to retrieve document data from a MongoDB database, using both M …
Learning Spark SQL String Functions with Explanations and Code Examples
The post Learning Spark SQL String Functions with Explanations and Code Examples appeared first on SQLServerCentral.
Exporting and Importing Registered Servers in SSMS
Many SQL Server Management Studio (SSMS) users connect to the same SQL Server instances over and over. In some organizations, the number of instances can be large, which makes flipping through a long list of instances in the Connect to SQL Server dialo …
Database technology learning resources
Before I started as the editor of Simple Talk, I worked on SQL Server. Only. (Ok, I used Redgate’s tools too). But when I started here, one of the goals was to stretch the topics farther and farther into more and more data platforms. And it is no …
Building RESTful APIs in Rust With Actix and Diesel
There are many packages and tools that you can use to facilitate your API development with Rust. Rust has a rich third-party ecosystem of crates for building APIs, including web packages like Actix and Rocket and ORMs like Diesel and SeaORM. This artic …
What happens when we drop a column on a SQL Server table? Where's my space?
Short answer: The column is marked as ‘deleted’ and will stop being visible/usable. But, most importantly – The record/table size will remain unchanged. A metadata operation Dropping a column is a metadata/logical operation, not a physical one. This me …
[Video] Database Answers in Hong Kong
As my time in Hong Kong came to an end, I sat inside on a foggy morning and hit your top-voted questions from https://pollgab.com/room/brento. Here’s what we covered: 00:00 Start 01:32 MyTeaGotCold: Are there any signs of brain drain from SQL Ser …
Working with IAsyncEnumerable in C#
IAsyncEnumerable is a powerful interface introduced in C# 8.0 that allows you to work with sequences of data asynchronously. It is a great fit for building ETLs that asynchronously stream data to get it ready for transfer. You can think of IAsyncEnumer …
Troubleshooting Mysterious Blocking Caused By sp_getapplock
I’m kinda weird. I get excited when I’m troubleshooting a SQL Server problem, and I keep hitting walls. I’ll give you an example. A client came to me because they were struggling with sporadic performance problems in Azure SQL DB, and …
It’s almost time for Fundamentals Week! Have you signed up yet?
The Monday after next, I’ll be getting together with dozens of smart people who are a lot like you. You’ve attended some of my free online streams, read my blog posts, and you use the First Responder Kit. You enjoy my laid-back, humorous ap …
Recursive CTEs: Transforming and Analyzing Data in PostgreSQL, Part 3
The first two articles in this series demonstrated how PostgreSQL is a capable tool for ELT – taking raw input and transforming it into usable data for querying and analyzing. We used sample data from the Advent of Code 2023 to demonstrate some o …
Unlocking Power with Updatable Views in SQL Server
Relational database management systems (RDBMS) rely on Structured Query Language (SQL) as their foundation for data management and manipulation. Views in SQL Server offer a strong way to abstract the underlying intricacy of data structures. While many …