Skip to content Skip to sidebar Skip to footer
Home / Beautifulsoup / Python / Web Scraping

Scraping Table With Beautifulsoup

Post a Comment
In this first code, I can use BS to get all the info within the table of interest: from urllib import urlopen from bs4 import BeautifulSoup html = urlopen('http://www.pythonscrapi

Solution 1:

When urllib.urlopen returns the content of a webpage, it returns the HTML from a URL with JavaScript turned off. In your case, this means that when urllib loads the relevant URL, the table with id="tournamentTable" never actually loads.

You can observe this behaviour by turning off JavaScript in your browser and loading the URL.

To scrape a webpage with content rendered by JavaScript you might want to consider using a browser automation package such as Selenium. If you scrape regularly you might also want to download a 'JavaScript switcher' plugin which allows you to toggle JavaScript on and off with ease.

Post a Comment for "Scraping Table With Beautifulsoup"