You need some kind of algorithm to classify that amount of text, and defining a set of keywords is not feasible for a lot of reasons.
You would need to have an idea of what the articles are talking about already, with so many posts keywords would certainly overlap in a lot of cases, and your categorization would be limited to words and not meaning: imagine a post about blogging. It will contain references to content marketing, SEO and possibly web-design. How can you determine programmatically what's the actual topic without reading the text?
Solution 1: an algorithm capable of finding intra-article keyword density with respect to the rest of the corpus, so excluding the most frequent words from the analysis (hint: if blogging appears in every article with the same frequency, is not so relevant for classification). You can realize something similar with TF-IDF (Term Frequency — Inverse Document Frequency) vectorizing. An implementation is possible with Sklearn and NLTK.
Solution 2: Use word embeddings to reduce dimensionality and cluster your articles on meaning. The approach is similar, but words are not considered in a vacuum: the approach above would not take into account if blogging and blog have the same meaning. The words are grouped if they tend to appear together in a document and the algorithm you are looking for is LSA (Latent semantic analysis), which can be implemented as a PCA (Principal component analysis) on Sklearn.
Solution 3: A Fill-Mask model based on BERT to assign a label to every article. You can fine-tune a model on Huggingface or use one as-it-is, but don't expect miracles.
Solution 4, not suggested: There are easy pipelines to set up on Spacy to extract keywords from text. Look for RAKE and you should find some more methods. It is simple to start with this one but for experience, I can tell you it is fairly inaccurate.
I hope that helps!
) .
