o ch+;@sdZddlmZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl m Z ddl m Z ddlmZddlmZmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlm Z!ddl"m#Z#m$Z$ddl%m&Z&ddl'm(Z(ddl)m*Z*ej+ej,dde GdddZ-d;ddZ.dd'd(Z0d?d-d.Z1d@dAd4d5Z2dBdCd8d9Z3e4d:kre3dSdS)Da Instagram Reels Scraper ======================= This script scrapes view statistics from the latest Reels of a public Instagram profile. It relies exclusively on browser automation using Selenium and does **not** call any official or unofficial APIs. The script scrolls through the account's Reels page, opens each reel in a new tab, extracts the number of views, the publication date, and the unique slug/ID from the URL, then writes the collected data to a JSON or CSV file. A running total of views across the inspected reels is also reported. Usage (from the command line):: python scrape_instagram_reels.py --username sinkmate.de --reels 20 --output output.json --profile-id 687e41cb8cfee375657ad010 --token your_token Options: --username Instagram username to scrape (required) --reels Number of latest reels to process (default: 20) --output Output filename, .json or .csv (required) --profile-id GoLogin profile ID (required) --token GoLogin API token (required) --delay Delay in seconds between actions to mimic human behaviour (optional) Requirements: - Python 3.8+ - Selenium >= 4.0 - GoLogin Python SDK - webdriver-manager Notes: * Uses GoLogin profiles for browser fingerprint management * No Instagram account is required to run this script; it uses only publicly available data. * Web scraping can break if Instagram updates their HTML or CSS. ) annotationsN) dataclass)datetime)Path)ListOptional)GoLogin) webdriver)By)Keys)Service) WebDriverWait)expected_conditions)NoSuchElementExceptionTimeoutException) ActionChains)ChromeDriverManager)Displayz[%(levelname)s] %(message)s)levelformatc@s<eZdZUdZded<ded<ded<ded<dd d Zd S)ReelInfoz8Container for information about a single Instagram reel.strurlintviewsz Optional[str]dateslugreturndictcCs|j|j|j|jdS)Nrrridrrrr)selfr#//var/www/html/scraper/scrape_instagram_reels.pyto_dictTs zReelInfo.to_dictN)rr)__name__ __module__ __qualname____doc____annotations__r%r#r#r#r$rKs rtextrrrcCs|dddd}td|}|sdS|\}}|dddd}zt|}Wn ty6YdSwd }|rL|}|d vrFd }n|d vrLd }t||S)uConvert a view count string (e.g. '12,3�Tsd. Aufrufe', '1.2M views') to an integer. The view counts on Instagram can be displayed in different locales and with abbreviations. This function normalises the number into an integer. Parameters ---------- text: str Raw text containing the view count and possibly a label ("Aufrufe", "views"). Returns ------- int Parsed integer view count. Returns 0 if parsing fails. u   z([\d.,]+)\s*([KMk]|Tsd\.|M|Mn)?r.,)ktsdi)mmni@B) replacestripresearchgroupsfloat ValueErrorlowerr)r+cleanedmatch number_strunitvalue multiplierr#r#r$parse_view_count]s&     rDdriverwebdriver.Remotedelayr;Nonec CsBtjdftjdftjdftjdftjdfg}|D]*\}}zt|dt||f}||r5t|Wn t t fyBYqwz2| tjd}| tjd}| tjd }| d | d | d ||rst|WdSWdSt t fyz| tjd  tj|rt|WYdSWYdSYYdSw)zAttempt to close cookie banner and login pop-up if present. Parameters ---------- driver: webdriver.Remote An instance of the Selenium WebDriver. delay: float Optional sleep delay to slow down actions. u//button[contains(translate(text(),'ABCDEFGHIJKLMNOPQRSTUVWXYZ���','abcdefghijklmnopqrstuvwxyz���'), 'allow') and contains(text(), 'Alle')]u~//button[contains(translate(text(),'ABCDEFGHIJKLMNOPQRSTUVWXYZ���','abcdefghijklmnopqrstuvwxyz���'), 'zustimmen')]u//button[contains(translate(text(),'ABCDEFGHIJKLMNOPQRSTUVWXYZ���','abcdefghijklmnopqrstuvwxyz���'), 'akzeptieren')]u//button[contains(translate(text(),'ABCDEFGHIJKLMNOPQRSTUVWXYZ���','abcdefghijklmnopqrstuvwxyz���'), 'optionale cookies ablehnen')]z///button[contains(text(), 'Allow all cookies')]z2//*[@id="loginForm"]/div[1]/div[1]/div/label/inputz2//*[@id="loginForm"]/div[1]/div[2]/div/label/inputz)//*[@id="loginForm"]/div[1]/div[3]/buttonz sinkmate.devzB9!n4g64zarguments[0].click();bodyN)r XPATHr untilECelement_to_be_clickableclicktimesleeprr find_element send_keysexecute_scriptTAG_NAMEr ESCAPE) rFrHcookie_selectorsbyselectorbuttonusername_fieldpassword_field login_btnr#r#r$handle_cookies_and_login_promptsF      r_ reels_countList[ReelInfo]c Csg}t}d}d}t||kr||kr|tjd}|D]}|d} | r)| |vr*q|| zE|tjd} | j} t | } | dkrst d| d| d| d |d t d |tjd} | j} t | } t d | dWntyt d | dd} Ynw| ddd} |t| | d| dt||krnq|dt d|d 7}t||kr||ks|d|S)zLScrolls and collects reels with URLs and view counts directly from the grid.r//a[contains(@href, '/reel/')]hrefzu.//div/div/div/span/span[contains(@class, 'x1vvkbs') and contains(@class, 'xdj266r') and contains(@class, 'x14z9mp')]uViews-Element f�r Reel z1 konnte nicht korrekt ausgewertet werden. Text: ''error_screenshot_z.pngr1uNächster Versuch: 'uViews-Element nicht f�r Reel z) gefunden. Vermutlich noch nicht geladen./Nr!z'window.scrollBy(0, window.innerHeight);)setlen find_elementsr rL get_attributeaddrSr+rDloggingwarningsave_screenshotrQrRrrstripsplitappendrrU)rFr`rH collected seen_urlsattempts max_attemptsreelsarspan views_textrrr#r#r$scroll_to_load_reelssL         'r}results total_views output_pathrc Cs|jdkr6dd|D}||d}|jddd}tj||dd d Wd d S1s/wYd S|jd kr{|jdd dd,}tj|gdd}||D] }|| qU|d|d d dWd d S1stwYd St d|j)a=Save the list of reel information to JSON or CSV. Parameters ---------- results: List[ReelInfo] List of extracted reel information. total_views: int Sum of all views across the processed reels. output_path: Path Destination file. The extension determines the format. z.jsoncSsg|]}|qSr#)r%.0rr#r#r$ sz save_results..)ryrwzutf-8)encodingFrJ) ensure_asciiindentNz.csvr/)newlinerr) fieldnamesTOTALzUnsupported output format: ) suffixr=openjsondumpcsv DictWriter writeheaderwriterowr%r<) r~rrdatapayloadfcsvfilewriterrr#r#r$ save_resultss" ""rusernamecountoutput profile_idtokenc Cs|pd}|pd}t||gdd}tddd} | z|} |} tt| d} t} | d | | d | d | d tj | | d }| ddt dd|d}td|||t|ddzt|dttjdfWntytd||| YWdSwtd|dt|||}t dd|D}tdt!|d|t"|}t#|||td|$||WdSt%y}ztd |z|WYd}~ww)!aMain entry point to run the reel scraper with GoLogin. Parameters ---------- username: str Instagram account username (without '@'). count: int Number of latest reels to process. output: str Path to output file (.json or .csv). delay: float Optional delay between actions to avoid detection. profile_id: str GoLogin profile ID (optional, uses hardcoded value if not provided). token: str GoLogin API token (optional, uses hardcoded value if not provided). 685054d57cb66888bd4a4e6fzeyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI2ODQzMzQ5MjIwMDMwYjQ0OTdlZmVmN2YiLCJ0eXBlIjoiZGV2Iiwiand0aWQiOiI2ODQ0NDA0MDYyNjQ0YWU1NzQ5ODI2MWMifQ.vj-RJv3gm0KpGnb0fAtmipDS2pDnLAFlDik8QKpsc8A)z --no-sandboxz--disable-dev-shm-usagez--enable-unsafe-swiftshaderz --disable-gpuz--disable-audio-output)rr extra_paramsr)ii8)visiblesize)driver_versiondebuggerAddressz--lang=en-US,en;q=0.9z---disable-blink-features=AutomationControlledz--disable-notifications)serviceoptionsiirJzhttps://www.instagram.com/z/reels/zOpening page: rcz\No reels found on the page. Check if the username is correct or if Instagram requires login.NzCollecting up to u reel links�css|]}|jVqdSN)rrr#r#r$ ^szrun_scraper..zFinished scraping z reels. Total views: zResults saved to zError during scraping: )&rrstartget_chromium_versionr rinstallr ChromeOptionsadd_experimental_option add_argumentChromeset_window_sizerQrRroinfogetrmove_by_offsetperformr rMrNpresence_of_element_locatedr rLrerrorquitstopr}sumrkrrresolve Exception)rrrrHrr PROFILE_IDTOKENgldisplaydebugger_addresschromium_versionrrrF start_urlr~rrer#r#r$ run_scraper sr               rargvList[str] | NonecCstjdd}|jdddd|jdtdd d |jd dd d|jd dd|jddd|jdtddd ||}|jdkrE|dt|j |j|j |j |j |j dS)NzMScrape the latest Instagram reels of a public profile using GoLogin profiles.) descriptionz --usernameTzInstagram username (without @))requiredhelpz--reelsz.Number of recent reels to scrape (default: 20))typedefaultrz--outputz Output file path (.json or .csv)z --profile-idzCGoLogin profile ID (optional, uses hardcoded value if not provided))rz--tokenzBGoLogin API token (optional, uses hardcoded value if not provided)z--delayrEz>Optional delay (seconds) between actions to mimic a human userrz"--reels must be a positive integer)argparseArgumentParserrrr; parse_argsryrrrrrHrr)rparserargsr#r#r$mainws    "r__main__)r+rrr)rE)rFrGrHr;rrI)rFrGr`rrHr;rra)r~rarrrrrrI)rENN)rrrrrrrHr;rrrrrrIr)rrrrI)5r) __future__rr contextlibrrror8sysrQ dataclassesrrpathlibrtypingrrgologinrseleniumr selenium.webdriver.common.byr selenium.webdriver.common.keysr !selenium.webdriver.chrome.servicer selenium.webdriver.support.uir selenium.webdriver.supportrrNselenium.common.exceptionsrrselenium.webdriverrwebdriver_manager.chromerpyvirtualdisplayr basicConfigINFOrrDr_r}rrrr&r#r#r#r$sP &               ( 3 4 k