Building a Data Pipeline - Languages and Stack
Choosing a language and stack
Intro
Hello! Welcome to the exciting topic of ingesting massive amounts of pastries data! We talked about ingest a bit in the previous tutorial; in this one, we’ll examine various architectures in great detail.
Let’s get the controversial part out of the way. Language. =)
Building a Data Pipeline - Part 0
Covers general elements of data pipelines, what they do, and why they do it
Intro When I started this blog, I wanted it to show building multiple different types of projects from a practical perspective. As an astute reader, I’m sure you’ve noticed that Iridium has been the only the project we’ve been working on. Time to change that!
Welcome to the first post of Project Grimwhisker!
(adsbygoogle = window.adsbygoogle || []).push({}); What? It’s a data pipeline.
Primer on Database Sharding
Covers what database sharding means and different implementations
Intro Another exciting database article! I bet you couldn’t wait! =) I’m going to keep this one short, and cover one topic: database sharding.
What Is It? Let’s say you have an application that needs a backend database. You’re using Postgres, and you’ve scaled it as much as you can vertically. Like, you’re using AWS X1E 32xlarge that cost around $20,000 USD per month. What can you do next?
Primer on NewSQL
Covers what NewSQL is and when to use it
Intro Note This article goes well with this primer on databases, so you may want to read that one first. If the thing most missing from your life is another type of SQL, then you’ve come to right article! I speak, of course, of NewSQL.
WTF is NewSQL? You get NewSQL when you take NoSQL (or a plain KV store) and put SQL on top to form a sort of unholy sandwich.
Primer on Database Types
Covers the major types of databases and when to use which type
Intro Hello! This article gives you an overview of the major database types, a summary of how they work, and when and why you should use one.
This is not a short entry. Sorry. =)
CAP Theorem You may have heard this term whispered amongst your colleagues. While it is an actual theorem, and an important one, people often take different meanings away after hearing about it.