Selenium print source code python

Here is a simple code all it does is it opens the url and gets the length page source and waits for five seconds and will get the length of page source again.

if __name__=="__main__":
    browser = webdriver.Chrome()
    browser.get("https://codefights.com/tournaments/Xph7eTJQssbXjDLzP/A")
    initial = len(browser.page_source)
    print(initial)
    time.sleep(5)
    new_source = browser.page_source
    print(len(new_source)

see the output: 15722 48800

you see that the length of the page source increases after a wait? you must make sure that the page is fully loaded before getting the source. But this is not a proper implementation since it blindly waits.

Here is a nice way to do this, The browser will wait until the element of your choice is found. Timeout is set for 10 sec.

if __name__=="__main__":
    browser = webdriver.Chrome()
    browser.get("https://codefights.com/tournaments/Xph7eTJQssbXjDLzP/A")
    try:
        WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, '.CodeMirror > div:nth-child(1) > textarea:nth-child(1)')))  # 10 seconds delay
        print("Result:")
        print(len(browser.page_source))
    except TimeoutException:
        print("Your exception message here!")

The output: Result: 52195

Reference:

https://stackoverflow.com/a/26567563/7642415

http://selenium-python.readthedocs.io/locating-elements.html

Hold on! even that wont make any guarantees for getting full page source, since individual elements are loaded dynamically. If the browser finds the element it moves on. So make sure you find the proper element to make sure the page has been loaded fully.

P.S Mine is Python3 & webdriver is in my environment PATH. So my code needs to be modified a bit to make it work for Python 2.x versions. I guess only print statements are to be modified.

This article is a part of our Content Hub. For more in-depth resources, check out our content hub on Selenium Python Tutorial.

Retrieving the page source of a website under scrutiny is a day-to-day task for most test automation engineers. Analysis of the page source helps eliminate bugs identified during regular website UI testing, functional testing, or security testing drills. In an extensively complex application testing process, automation test scripts can be written in a way that if errors are detected in the program, then it automatically.

• saves that particular page’s source code.
• notifies the person responsible for the URL of the page.
• extracts the HTML source of a specific element or code-block and delegates it to responsible authorities if the error has occurred in one particular independent HTML WebElement or code block.

This is an easy way to trace, fix logical and syntactical errors in the front-end code. In this article, we first understand the terminologies involved and then explore how to get the page source in Selenium WebDriver using Python.

TABLE OF CONTENT

• What Is An HTML Page Source?
• What Is An HTML Web Element?
• How To Get Page Source In Selenium WebDriver Using Python?
• How To Retrieve HTML Source Of WebElement In Selenium?
• How To Retrieve JSON Data From An HTML Page Source In Python Selenium WebDriver?

What Is An HTML Page Source?

In non-technical terminology, it’s a set of instructions for browsers to display info on the screen in an aesthetic fashion. Browsers interpret these instructions in their own ways to create browser screens for the client-side. These are usually written using HyperText Markup Language (HTML), Cascading Style Sheets (CSS) & Javascript.

This entire set of HTML instructions that make a web page is called page source or HTML source, or simply source code. Website source code is a collection of source code from individual web pages.

Here’s an example of a Source Code for a basic page with a title, form, image & a submit button.

What Is An HTML Web Element?

The easiest way to describe an HTML web element would be, “any HTML tag that constitutes the HTML page source code is a web Element.” It could be an HTML code block, an independent HTML tag like
, a media object on the web page – image, audio, video, a JS function or even a JSON object wrapped within tags.

In the above example –