I've been considering switching from xata.io to aiven.io because they add special "xata" columns to your tables and I had doubts on how to export import the data... or how to work around that.
I need an agile way of maintaining the database... So I've improved my Purge.js script and it now can handle 30 threads without throwing a single network error while doing requests. It just do HEAD requests, on 404 the rows are deleted.
I found that I can just work on my local database, then export my data to a CSV file... and import the data straight to xata, I've imported 94271 rows with 0 errors. Fast enough & Simple.
Now, my Next Js site can load my scraped content from my xata.io database
![ZnIHwho[1].png ZnIHwho[1].png](https://www.blackhatworld.com/data/attachments/750/750920-89b6e5aa54b87fb4897d85cd23151fbe.jpg?hash=Mud_hTtrzP)
I need an agile way of maintaining the database... So I've improved my Purge.js script and it now can handle 30 threads without throwing a single network error while doing requests. It just do HEAD requests, on 404 the rows are deleted.
I found that I can just work on my local database, then export my data to a CSV file... and import the data straight to xata, I've imported 94271 rows with 0 errors. Fast enough & Simple.
Now, my Next Js site can load my scraped content from my xata.io database
![ZnIHwho[1].png ZnIHwho[1].png](https://www.blackhatworld.com/data/attachments/750/750920-89b6e5aa54b87fb4897d85cd23151fbe.jpg?hash=Mud_hTtrzP)