vittorianicolosi1994
Regular Member
- Jan 17, 2024
- 279
- 81
code:[
!pip install -U crawl4ai
!pip install nest_asyncio
# Check crawl4ai version
import crawl4ai
print(crawl4ai.__version__.__version__)
##### Setup Crawl4ai
The following command installs Playride and its dependencies and updates a few configurations for Crawl4ai. After that, you can run the doctor command to ensure everything works as it should.
%%capture
!crawl4ai-setup
!crawl4ai-doctor
# If you face with an error try it manually
# !playwright install --with-deps chrome # Recommended for Colab/Linux
I suggest you first try the code below to ensure that Playwright is installed and works properly.
import asyncio
import nest_asyncio
nest_asyncio.apply()
from crawl4ai import AsyncWebCrawler, CacheMode, CrawlerRunConfig
async def fetch_html():
url = "https://www.duelingbook.com/deck?id=12928546" # Replace with your target URL
config = CrawlerRunConfig(cache_mode=CacheMode.BYPASS)
async with AsyncWebCrawler() as crawler:
result = await crawler.arun(url=url, config=config)
print("Raw HTML Content:")
print(result.html) # Directly access .html attribute
# Run the crawler
asyncio.run(fetch_html())
]
after i compare with entire html source and it isnt the same .crawl4ai ouptput only visible text
!pip install -U crawl4ai
!pip install nest_asyncio
# Check crawl4ai version
import crawl4ai
print(crawl4ai.__version__.__version__)
##### Setup Crawl4ai
The following command installs Playride and its dependencies and updates a few configurations for Crawl4ai. After that, you can run the doctor command to ensure everything works as it should.
%%capture
!crawl4ai-setup
!crawl4ai-doctor
# If you face with an error try it manually
# !playwright install --with-deps chrome # Recommended for Colab/Linux
I suggest you first try the code below to ensure that Playwright is installed and works properly.
import asyncio
import nest_asyncio
nest_asyncio.apply()
from crawl4ai import AsyncWebCrawler, CacheMode, CrawlerRunConfig
async def fetch_html():
url = "https://www.duelingbook.com/deck?id=12928546" # Replace with your target URL
config = CrawlerRunConfig(cache_mode=CacheMode.BYPASS)
async with AsyncWebCrawler() as crawler:
result = await crawler.arun(url=url, config=config)
print("Raw HTML Content:")
print(result.html) # Directly access .html attribute
# Run the crawler
asyncio.run(fetch_html())
]
after i compare with entire html source and it isnt the same .crawl4ai ouptput only visible text