I got to implement cookie management next. Now following is the garbage I got off AI.
wtf... time to work
"I think what distinguishes human beings from all the other creatures on the earth is... is... our ability to engineer." - Michael Saylor
What am I doing exactly?
I have 15 - 20% finished project and it got chaotic. I'm rewriting this by copy-pasting and editing parts + building a mini framework based on that project to make it less chaotic. Then reuse that framework in other projects.
Edit:
We take things like "index.ts" or "index.js" for granted and assume we know what they're for.
But they're actually useful for organization.
We look at index like we would as in library. Index is as "(in a book or set of books) an alphabetical list of names, subjects, etc. with reference to the pages on which they are mentioned."
Instead of doing exports inside every file, we use index.ts for that.
Now we export everything just from "index" and import from "index" instead of individual files.
You can still use "export" keyword inside files, just to make it clear it'd be exported. You can export one thing multiple times from many places which is a sort of a trick.
I have 4 hours till my next shift, so I work on this now as yesterday I was unable to focus due to eating all day long and family meeting + trolling on the forum for hours.
There's something special during xmas when everyone just can't focus.
Many known workaholics were inactive during that time. So was I (success compared to previous years).
I wasn't drinking though which won't force me to spend another week for recovery.
Priority - list of to-do (to find order in chaos):
1. Finish cookie management off file. Off database isn't useful now.
2. Make git commits and push all the stuff to repository once it's in "working version".
I don't really know what are all benefits to it but I noticed sometimes I need to see how it was before a change. Naturally you would like to see how something worked in the previous version.
3. Write some tests to practice writing tests.
4. Create testing environment by writing decoupled functions that can be run using nothing else than cookies loaded off file.
5. Implement mechanisms to abstract classes that classes inherit from to check whether account works or not before doing anything.
6. Clean up and split browser.ts file - it's too long as it's 250 lines and all it does is opening browsers. It does some things that could be put away into different modules.
On first step - Finish cookie management off file
I'll import cookies from json. So I'll import json. For now I can export them from browser using "Export cookie Json" chrome addon.
Getting JSON as "import" can be done as here https://www.typescriptlang.org/docs/handbook/release-notes/typescript-2-9.html#new---resolvejsonmodule
"import * as TestCookies from "./test-cookies/index.json with { type: "json" };";" once I set "resolveJsonModule": true," and "
"module": "ESNext" in tsconfig.json.
Turns out "number" type in typescript is also something like this "1743609864.452692". In other programming languages it wasn't called number but float or some.
page.setCookie(...cookies) - this apparently accepts "(...cookies)" which is called "https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Spread_syntax" - basically that function will accept any number of arguments.
When I want to reuse some function, I want to create a reusable function with common name that I can use multiple times later and edit once, to edit everywhere.
So I create a module / file called "links.ts". And in that file I put:
Then I can import "goToYouTube" and use as "await goToYouTube(page) wherever I want based on that "goToWebsite" function. Very short, right.
For now I have this type of thing that seems to work with "any" types but I don't care now.
JavaScript:
import * as TestCookies from "./test-cookies/index.json" with { type: "json" };
import { Cookies, Cookie } from "../database/models";
import { Page, Cookie as PuppeteerCookie } from "puppeteer";
export async function loadCookiesFromFile(page: Page) {
const cookiesToLaunch = TestCookies;
try {
const cookies: any = cookiesToLaunch;
const cookiesDefault: any = cookies.default;
console.log(cookiesDefault);
await page.setCookie(
...cookiesDefault.map((cookie) => ({
name: cookie.name,
value: cookie.value,
domain: cookie.domain,
path: cookie.path,
expires: cookie.expires,
httpOnly: cookie.httpOnly,
secure: cookie.secure,
}))
);
} catch (error) {
console.error("Error loading cookies from file:", error);
}
}
Later homie...
On commits
Did some commits using "source control" tab in Visual Studio Code. Each commit is one significant change, so it's actually useful to describe and it will work as a list of tasks later.
We can use it to brag and build personal brand showing off to other people.
I need to find a scheduling thing (npm module) to put things away in time as that's required to not fire my stuff early.
I have a list of sets of operations and between each operation there's is a random delay.
Now between each set of operations there is a long delay ie. 6 - 48 hours.
So it's not a loop but a long, spontaneous schedule.
Each set it constructed of random operations.
Sets of operations are put in random order as well.
Based on some operations there are other operations in the future but there might be a problem that the browser will be busy and 2 operations would overlap. So they need to be spaced in time and done subsequently.
What library is good for that? Probably none as 90% is custom anyways.
Maybe I'll find some article on scheduling tasks in node.js.
Edit: Agenda and other libraries look fine. But which one will be comfortable for me? I'll check their docs.
This code is all over the place. And honestly, the way you've (or whatever AI you're using) written it makes it really hard to follow. It's like there's no order or structure at all in your code and your thoughts/wall of text in this topic.
One glance and I see you're setting cookiesDefaultas any, but then in loadCookiesFromFile you're mapping over the values. Code makes no sense and is very prone to bugs.
JavaScript:
const cookiesDefault: any = cookies.default;
But you do you. Props for trying something
For scheduled stuff use https://www.npmjs.com/package/cron
Code is not clean.
You will barely understand what's happening messing with import and exports when project grows.
Also for scaling, try to be minimal as possible and scale it with containers (docker/swarm or k8s)
Do micro-services architecture
Just use "fs.promises" to async read files
Use postgres as database or dont use it at all if the overhead is not worth it (for the amount of data you're storing)
If data is going back and forward from db and its not large, consider replacing it with redis (in memory cache) as you will gain tremendous performance
This might not work due to business reasons. It all happens on Windows.
Do micro-services architecture
Just use "fs.promises" to async read files
Use postgres as database or dont use it at all if the overhead is not worth it (for the amount of data you're storing)
If data is going back and forward from db and its not large, consider replacing it with redis (in memory cache) as you will gain tremendous performance
It's ok! Every self-taught developer started somewhere.
You could scale it up using nodejs cluster if your machine is multi-core, just make sure its stateless to not issue into any race conditions.
SQL these days is not worth learning (if you dont need very specific use case / extreme performance), there are so many ORMs and you can just use them. Try "prisma".
Look into being stateless, save you application state externally and safe from multiprocessing so you can scale very easily.
Use queues for processing tasks, it will make everything easier.
It's ok! Every self-taught developer started somewhere.
You could scale it up using nodejs cluster if your machine is multi-core, just make sure its stateless to not issue into any race conditions.
SQL these days is not worth learning (if you dont need very specific use case / extreme performance), there are so many ORMs and you can just use them. Try "prisma".
Look into being stateless, save you application state externally and safe from multiprocessing so you can scale very easily.
Use queues for processing tasks, it will make everything easier.
Lol imagine this level of reply on Reddit. BHW surprised me again.
Though it's a marketing forum, they give decent programming advice.
I had something like queues on my mind, but I don't have any experience with writing them other than little bit Laravel.
That was long time ago.
I have never done anything stateless.
"When a program "does not maintain state" (is stateless) or when the infrastructure of a system prevents a program from maintaining state, it cannot take information about the last session into the next, such as settings the user chose or conditions that arose during processing."
So maybe I have, just don't realize because this term is vague.
So I'm using anti detect browser which means my program isn't really capable of being scaled forever with some kubernetes.
I have never seen anyone successful with this approach. I feel like people think of scaling programs too much and not money. Technical implementation here is 10% of work. The rest is solving business problems.
It's not a project for client.
It's supposed to work, not look pretty on paper with some "this is scalable!! Pay attention" red sign in the middle.
Though if you send me guides I can apply those. I don't care about typescript for random functions and being precise with types in every single place.
It's more needed for data structures and classes, interfaces, inheritance etc. I'll write docs later.
I'm solo and working daily on many things.
The chaotic style of writing classes lets me rest when I'm tired of implementation of one class and meanwhile I can work on another. This way I'm easier on myself.
I tried to code one class and solve a technical problem no matter what, making things look perfect, but it just was causing harm. Zooming in so hard on one little class is just pointless from energy expenditure perspective. Especially if it's something as simple as cookies setup.
Try to solve math problems in a single session without longer breaks between trials when you do it for the first time. This would burn! And I remember how frustratiing it was.
The rule is it works, it's good. Tests will confirm.
Well, why don't I just outsource it? I don't know. Maybe I should. Maybe next time. For now I can still learn.
Maybe I'll set typescript mode to stricter to get disciplined.
It's complete piece of garbage that no sane person should consider using because all it does is it introduces them to corporate level nonsense solutions that maybe will work in SaaS on a 256 core machine but would never work on Windows for small scalers aiming for $3k revenue per day.
1. Redis.
2. Linux.
3. Stupid library that isn't really that good but is quite cheap (marketing part), so st00pid *rich* developers ($15k mo payment) buy extra PRO license for their BOSS money.
So what I'll be doing is creating my own scheduler, queue, rate limiting, job overlapping avoidance solution system.
Please, don't do anything related to software in 2024 if you think someone has solved problems for you.
These idiots pissed me off so hard I would write 100000 pages describing their madness.
I literally feel inflammation increasing in my body just looking at this idiotic field. It's probably the most moronic filled with bs products engineering field ever created.
So stupid bullshit, millions of tightly coupled dependencies. Example: Scheduler > coupled wth Redis > coupled with Linux > coupled with Linux browsers / useragent.
Rationalization to switch to linux? "You can use Docker bro".
Revenue and source of income: my boss pays me. I'm changing my company every 2 years... Each time they say "project failed" but I have no idea why!
Yes, you can afford this "LInux" dependency because you have no fcking responsibility to generate revenue. You only lie to your boss it will work.
The chaotic style of writing classes lets me rest when I'm tired of implementation of one class and meanwhile I can work on another. This way I'm easier on myself.
Interesting rant, I agree this field can be really daunting.
Tons of stuff to learn as a solo developer trying to create some hustles, while the peers you're interacting with are often employed by a relaxed company which affords them the ability to study (and engineer) overly-complex stuff without having to accomplish stuff with it quickly.
"Well no it's not actually a bloated overly complex mess abstracting everything behind 5 layers needlessly!!!! It only took me a full week of studying to figure it out in my cubicle while collecting my salary!"
This is what happens when you happen to happen to make something happen for the first time.
You put a large file called "ActionService.ts" and you put everything in there creating sort of a large procedure that sucks and is pain in the ass to debug and everything feels like from decoupled pieces you created one messy coupled piece making you nauseous to work through it.
This would not be the way to earn a decoupling medal for sure.
663 lines long procedure... nothing of that makes sense.
Time to rework all of that garbage and create what should be created long time ago.
DB operations separated from automation operations separated from things that glue both of them together.
Then separation of something else I forgot about. Ahh, separation of scheduler and pieces that glue together scheduler and automation operations, and database operations.
Then separation of anti-detect browser APIs, so they can be easily implemented and switched back and forth.
You want these red links removed and make it have only one link - the green one.
If you are fast, cheap, you'll create red ones (many) which won't make you stand out in a good way.
A fckton of software is written like red ones. Why? Well, money... pressure...
Some software is written with red ones multiplied by x50. I worked like that for a while and all I wanted to do was alcohol / illicit drugs.
Boils down to organization - something an average lady is better at than an average man.
Women are known for striking the right balance between their personal and professional life.
So stupid bullshit, millions of tightly coupled dependencies. Example: Scheduler > coupled wth Redis > coupled with Linux > coupled with Linux browsers / useragent.
Stop shitting on Linux because you cannot use it / do not want to learn.
For software development it's way better due to all software made ad hoc for production, huge workload handling, security, better file system, stability ecc.
It runs over 90% of internet servers, and it's not popular on consumers because was made for servers not client. Even BHW website runs on Linux.
Also if you are on windows, just use WSL, and you have linux on your local machine with very small effort.
Who cares about Linux to windows when literally using nodejs that's crossplatform?
You should run the software in a remote server and only control it from your local windows. So you don't need any dependecy.
If you start using it for development you will not come back.
This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
By continuing to use this site, you are consenting to our use of cookies.
