Erik on Software

Sign in Subscribe

Topic

data

A collection of 171 issues

Media Query Source: Part 34 - MSDynamicsWorld (US digital magazine); Azure Data Box is counterpart to AWS Snow product family

The responses I provided to a media outlet on January 27, 2022: Media: What are the biggest gaps in Data Box capabilities, if any? Gfesser: The Azure Data Box family of products is essentially used to get data into Azure, focusing on storage optimized hardware. In contrast, the AWS Snow

Community Comment: Part 11 - It's common for company stakeholders to trivialize data architecture, often due to lack of understanding

The comments I provided in reaction to a community discussion thread. Principal at Data Analytics Firm: Two months ago I saved a startup client $15,000 a year on their Snowflake bill by simply sorting two of their BIG multi TB sized log data tables, changing around ETL jobs so

Community Comment: Part 10 - The difference between "none" and "N/A" (not applicable)

The commments I provided in reaction to a community discussion thread. Senior Data Scientist at Amazon: The goal is never to avoid failure, but to try enough so you can win. What are you trying to achieve for the last quarter of 2021? Principal Software Architect at Continuous Identity &

Community Comment: Part 9 - The "data is oil" expression is overused, but "sunshine" isn't a good "oil" substitute

The comments I provided in reaction to a community discussion thread. CEO of Insights & Data at Global Technology Consultancy: Data isn't oil. At least not anymore. Data is more like sunshine, abundant and unlimited in its potential. It has a positive impact on the environment. And critically,

Media Query Source: Part 33 - TechTarget (US digital magazine); Hadoop vs Spark: An an-depth big data framework comparison

* TechTarget (US digital magazine) * Apache Hadoop & Apache Spark * Overlap can exist between installations * Performance & use cases My responses ended up being included in an article at TechTarget (May 7, 2021). Extent of verbatim quote highlighted in orange, paraphrased quote highlighted in gray. Above image from cited article.

Community Comment: Part 8 - Data analysis isn't needed if the data is already understood

The comments I provided in reaction to a community discussion thread. Head of Experimentation at Social Media Platform: We should re-title “data cleaning” as “understanding the data”. Why? 1. It’s not a bad thing to spend 80 percent or more of our time deeply understanding the data. 2. Cleaning

New Book Review: "Learning Spark (Second Edition)"

New book review for Learning Spark (Second Edition), by Jules S. Damji, Brooke Wenig, Tathagata Das, and Denny Lee, O'Reilly, 2020: Copy provided by Databricks at Spark + AI Summit 2020. The foreword and preface to this book comment that an update to the first edition, published in 2015,

Media Query Source: Part 32 - TechTarget (US digital magazine); Do you need a data warehouse for business intelligence?

* TechTarget (US digital magazine) * Business intelligence & data warehouses * Usage of public cloud versus on premises * Cost is only one factor to consider My responses ended up being included in an article at TechTarget (February 12, 2021) entitled "Do you need a data warehouse for business intelligence?". Above

Community Comment: Part 6 - Don't use the term "tumbling interval" when the tech industry has already adopted the term "tumbling window"

The comments I provided in reaction to a community discussion thread. Senior Software Engineer (ex-Facebook, ex-Netflix): A lot of data engineers think coding in SQL is “beneath” them. They’re like, “I only use the Spark data frame API because that’s what all the cool engineers are doing nowadays”

Mentoring a Data Engineering Team while Running

A top 10 (US) grocer client contacted SPR to help them with their data lake. While we typically implement such solutions from scratch for our clients, the non-US division of this client had already put a solution in place, and the US division with which we worked was dependent on