How to scrape Youtube Video URLS based on Keywords?

dinesh1625 said:

Yeah that's what I am trying to do.

Click to expand...

dinesh1625 said:

Thanks for the info. I will check Scrapebox.

Click to expand...

thetrustedzone said:

if you have scrapebox you could do it like this :

site:youtube.com keyword1
site:youtube.com keyword2

Click to expand...

Ranking Expert said:

This is the best way.

Click to expand...

GoDesain said:

this sample with python without scrolling emulation..

Python:

import os
import requests
import urllib.parse
import json


class YtSearch:
    def __init__(self, search_terms: str, max_results=None):
        self.search_terms = search_terms
        self.max_results = max_results
        self.videos = self.search()

    def search(self):
        encoded_search = urllib.parse.quote(self.search_terms)
        BASE_URL = "https://youtube.com"
        url = f"{BASE_URL}/results?search_query={encoded_search}"
        response = requests.get(url).text
        while 'window["ytInitialData"]' not in response:
            response = requests.get(url).text
        results = self.parse_html(response)
        if self.max_results is not None and len(results) > self.max_results:
            return results[: self.max_results]
        return results

    def parse_html(self, response):
        results = []
        start = (
            response.index('window["ytInitialData"]')
            + len('window["ytInitialData"]')
            + 3
        )
        end = response.index("};", start) + 1
        json_str = response[start:end]
        data = json.loads(json_str)

        videos = data["contents"]["twoColumnSearchResultsRenderer"]["primaryContents"][
            "sectionListRenderer"]["contents"][0]["itemSectionRenderer"]["contents"]

        for video in videos:
            res = {}
            if "videoRenderer" in video.keys():
                video_data = video.get("videoRenderer", {})
                res["id"] = video_data.get("videoId", None)
                res["title"] = video_data.get("title", {}).get("runs", [[{}]])[0].get("text", None)
                results.append(res)
        return results

    def to_dict(self):
        return self.videos


def main():
    kwd = str(input('Enter keyword: '))
    results = YtSearch(kwd, max_results=20).to_dict()
    for x in results:
        uri = 'https://www.youtube.com/watch?v='+str(x['id'])
        title = str(x['title'])
        print(title, uri)


if __name__ == '__main__':
    main()

you can use youtube-dl also...

Click to expand...

function YouTubeScraper() {
  var sh1 = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Sheet1");
  var keyword = sh1.getRange("B1").getValue();
 
  var results = YouTube.Search.list('id,snippet', {q:keyword, maxResults:50});
//Video ID    Published Date    Channel ID    "Video Title
//"    Description    Thumbnail URL    Channel Title
  var items = results.items.map(function(e){
    return [e.id.videoId,
            e.snippet.publishedAt,
            e.snippet.channelId,
            e.snippet.title,
            e.snippet.description,
            e.snippet.thumbnails["default"].url,
            e.snippet.channelTitle]
  })
   sh1.getRange(4, 1, items.length, items[0].length).setValues(items)       
        
}

Ranking Expert said:

This is the best way.

Click to expand...

dinesh1625 said:

Thanks for the info. I will check Scrapebox.

Click to expand...

thetrustedzone said:

Something to mention better to use bing.com instead of google, bing has site: operator but they are not nazi like google regarding scraping they will not ban your proxies too fast like google.

Click to expand...

GoDesain said:

this sample with python without scrolling emulation..

Click to expand...

podninja said:

Hi All,

Please recommend me a tool or method to scrape youtube videos based on keywords. Please don't say do it manually

Thanks

Click to expand...

bhcg101 said:

I'd have to agree.. what do you want from the scrape. That would help to isolate a tool for this. For example.. would you be looking for expired domains from links in the youtube videos descriptions ? Your question is super broad.

Click to expand...

augustwu said:

That’s what I am doing everyday
Looks for expired domain , but it’s pretty hard to find good ones

Click to expand...