A date dimension for SQL Server
A date dimension for SQL ServerThe most common table you will find in a data warehouse will be the date dimension. There is no "right" implementation beyond what the customer needs to solve their...
View ArticleClik here to view.
Reading Excel files without Excel
Reading Excel files without ExcelA common problem working with Excel data is Excel itself. Working with it programatically requires an installation of Office, and the resulting license cost, and once...
View ArticleBiml Excel Data Source without Excel
Biml Excel Meta Data Source without ExcelIn the previous post, Reading Excel files without Excel, I showed some simple code to consume Excel without having Excel installed on your machine. How/Why...
View ArticlePolling in SQL Agent
Polling in SQL AgentA fun question over on StackOverflow asked about using SQL Agent with SSIS to poll for a file's existence. As the comments indicate, there's a non-zero startup time associated with...
View ArticleClik here to view.
SQL Server Agent Job Sort Order
SQL Server Agent Job Sort OrderToday's post could also be titled "I have no idea what is happening here." We have an agent job, "Job - Do Stuff". We then created a few hundred jobs (templates for the...
View ArticleMy github repository
In preparation for my talk at the Kansas City SQL Server User Group this afternoon, I am putting this post here so people can get the materials easily. Lightning Talks
View ArticleBiml in Azure aka CallBimlScriptContent
CallBimlScriptContent was introduced with the migration from Mist to BimlStudio. Why is this cool? You do not have to use files sitting on your computer as the source for your Biml. As long as you can...
View ArticleUsing Newtonsoft.Json with Biml
Using Newtonsoft.Json with BimlTwitter provided an opportunity for a quick blog post #sqlhelp #biml I would have the metadata in a Json structure. How would you parse the json in the C# BIML Script? I...
View ArticleGenerating characters in TSQL
Generating characters in TSQLI had to do a thing* and it involved generating "codes" as numbers were too hard for people. So, if you have need to convert an arbitrary number into characters, this is...
View ArticleMaking a delimited list
Making a delimited listThere are various ways to concatenate values together. A common approach I see is that people will add a delimiter and then the value and loop until they finish. Then they take...
View ArticleReusing your python code
Reusing your ptyhon codeI learned python in 2003 and used it for all the ETL work I was doing. It was beautiful and I would happilly wax to any programmer friends about the language and how they should...
View ArticleIncluding a local python module
Including a local python moduleAs we saw in reusing your python code, you can create a python file, a module, that contains our core business logic and then re-use that file. This post is going to talk...
View ArticleClik here to view.
ETL pattern for API source
ETL pattern for API sourceThe direction for software as a service providers is to provide APIs to access their data instead of structured file exports. Which is a pity, as every SaaS system requires a...
View ArticleClik here to view.
Extracting queries from SSIS packages
Extracting queries from SSIS packagesWe received an abomination of an SSIS package from a third party. It was a way to write a package that I don't think I would have suggested. Our job was to rewrite...
View ArticleClik here to view.
SSIS Azure Feature Pack and the Flexible File components
SSIS Azure Feature Pack and the Flexible File componentsThe Azure Feature Pack for SSIS is something I had not worked with before today. I have a client that wants to use the Flexible File...
View ArticleADF and MySql.Data.MySqlClient.MySqlException,Message=Got a packet bigger...
Azure Data Factory, ADF, and exception MySql.Data.MySqlClient.MySqlException,Message=Got a packet bigger than 'max_allowed_packet'My StackOverflow developer profile specifies "I'd prefer to not work...
View ArticleGENERATE_SERIES is a persnickety little function
GENERATE_SERIES is a persnickety little functionNew in SQL Server 2022 is the GENERATE_TIMESERIES table valued function. It's a handy function to generate a series of numbers. Once you understand why a...
View ArticleClik here to view.
Counting a character in a column
Counting a character in a columnI ran into an issue today that I wanted to write about so that maybe I remember the solution. We ran into a case where the source data in a column had an unprintable...
View ArticleFormatting a date in SQL Server with the Z indicator
Formatting a date in SQL Server with the Z indicatorIt seems so easy, I was building json in SQL Server and the date format for the API specified it needed to have 3 millsecond digits and the zulu...
View ArticleClik here to view.
ADF ForEach Activity
ADF ForEach ActivityThis occasional series of posts I'm calling Notes from the field is primarily for my benefit as I don't work with Azure Data Factory with any regularity and have no muscle memory in...
View ArticleSparkSQL Databricks [INVALID_USAGE_OF_STAR_OR_REGEX] Invalid usage of '*' in...
SparkSQL Databricks Error in SQL statement: AnalysisException: [INVALID_USAGE_OF_STAR_OR_REGEX] Invalid usage of '*' in expression `alias`.Hi, it's me. I'm the problem Dear self, when you recieve the...
View ArticleClik here to view.
Difference between SparkSQL and TSQL casts
Yet another thing that has bitten me working in SparkSQL in Databricks---this time it's data types. In SQL Server, a tinyint ranges from 0 to 255 but both of them allow for 256 total values. If you...
View ArticleADF, DB2 and unexpected token query error
ADF, DB2 and unexpected token query errorIn yet another exciting episode of head to desk, I have a beautiful DB2 query. It runs just fine in a tool like DBeaver. SELECT STR.CPY_CD , STR.STR_CD ,...
View ArticleDatabricks sparksql escaping quote/tick
If I had to embed a single quote in a query in TSQL, I would double it. In SparkSQL, I escape it like a classic C style string. So, the following shows how one would generate a query that is a query to...
View ArticleClik here to view.
Databricks sparksql concat is not your SQL Server concat
Databricks sparksql concat is not your SQL Server concatOne of these is not like the other... Databricks ConcatSQL Server ConcatThe concat function is super handy in the database world but be aware...
View Article