Where can store gigantic databases?

paalmtree · May 27, 2023

Terabytes of data... Where would one store it for cheap broken down in thousands of Excel files?

Gogol · May 27, 2023

Use a SQL variant. Mysql in my personal opinion. Vitess, or outsource it (e.g. planetscale. Just watch your queries), thank me later.
It depends though on what you mean by "data". I am assuming text data.

hennessy · May 27, 2023

If you want to actually use the data, I'd throw it into MongoDB. One node can easily handle 5TB, and can handle queries against collections containing billions of documents.

paalmtree · May 27, 2023

hennessy said:
If you want to actually use the data, I'd throw it into MongoDB. One node can easily handle 5TB, and can handle queries against collections containing billions of documents.

Well, it IS in Mongo now, but I am not a programmer. I just need to turn it all in usable Exel files that are stored somewhere for super cheap and easily downloaded and updated.

eQuick IT · May 27, 2023

Storing terabytes of data broken down into thousands of Excel files for cheap can be achieved by utilizing cloud storage providers like Amazon S3, Google Cloud Storage, or Microsoft Azure Blob Storage, which offer scalable and cost-effective storage solutions.

hennessy · May 28, 2023

How's your internet speed? Are you against getting some big SSDs or even a home server?

You can go cloud, but you'll pay a bit for egress, especially if you ever want to move the data to another service.

businessing · May 28, 2023

A Hetzner Storage Box... Cheap and virtually unlimited bandwidth

Where can store gigantic databases?

paalmtree

Regular Member

Gogol

Administrator

hennessy

Junior Member

paalmtree

Regular Member

eQuick IT

Registered Member

hennessy

Junior Member

businessing

Newbie

Main Menu

Marketplace

Making Money

BlackHat World