libs.deviantartapi module#

class libs.deviantartapi.SeleniumScrapper(username=None, password=None)[source]#

Bases: object

Parameters:

username (str) – Deviant art username
password (str) – Deviant art password

Returns:

None

Initializes the driver,Logs in to the account and loads the page
Uses cookies to login to the account if the cookies are present
If the cookies are not present, it will login to the account and save the cookies

get_deviant_links(base_url: list, next_page: int = 2) → set[source]#

Parameters:

base_url (list) – list of Deviant art Base urls
next_page (int) – Number of pages to scrape

Returns:

set of links to the deviant art pages

Return type:

set

Note

Need to enable Paging in Account settings -> Browsing -> Paging -> click through pages

Loads the page, Finds all the links to the deviant art pages
Then next page cursor is searched and the function is called again with the next page url
each page returns 24 links, then 24*next_page links will be returned
so if nexpage is 2 then 48 links are returned
Next page cursor count can be set manually in the function
Returns the set of links

login()[source]#

Returns:: None

Logs in to the account and saves the cookies

scroll(scroll_pause_time: float = 1.5) → str[source]#

Parameters:: scroll_pause_time (int) – Time to wait between scrolls
Returns:: Page source
Return type:: str

Warning

CURRENTLY NOT USED This function returns 24 links per page, even if there are more than 24 links in the page needs to be fixed in the future

Note

Need to enable scroll Account settings -> Browsing -> Paging -> Scroll through pages

Scrolls the page to load all the images
Can override the default time to wait between scrolls
Gets the scroll height and scrolls to the bottom of the page