I’m continuing the theme of #Hacktoberfest this week with another link on contributing to open source down the bottom. Beyond that I’ve been working on some Azure Databricks Delta lake basics to help with explaining it to others. Thanks to Paul Andrews post below I’ve also been looking back at data factory best practices too. Thanks Paul.
Interestingly I found it quite hard to find some “getting started” reference info on Delta Lake. The video below from Denny Lee at Databricks is good but if you know of any other resources you’d recommend, let me know in the comments.
Here’s what I’ve been reading and watching this week:
Simplify and Scale Data Engineering Pipelines with Delta Lake#
The first half of this video is some good theory around the “why” and there’s some nice reminders of data lake best practices too then Denny dives into the demos. Well worth a watch.
BEST PRACTICES FOR IMPLEMENTING AZURE DATA FACTORY#
by Paul Andrew
Thanks to Paul for keeping this one up-to-date. Having only just found this post, it was good to read through Paul’s best practices and how they align with my own. Having that confirmation of the same approach reinforces your thinking and helps solidify that best practice in the community. Paul has said it himself recently, that best practices are built in the community as products mature.
https://mrpaulandrew.com/2019/12/18/best-practices-for-implementing-azure-data-factory/
How anyone can contribute to the MS Docs website!#
Continuing the theme of open source for Hacktoberfest this month, I found this great discussion/demo from the Research Triangle PowerShell Users Group. It’s a bit long but if you want to skip to the demo part and how you can contribute, jump to 31:09 though the full video is worth a watch for some good PowerShell chat. Thanks to Paul for the link to this one via LinkedIn.
https://youtu.be/0_DEB61YOMc