any other alternative to selenium wire? /u/cope4321 Python Education

i’m running a kohls website scraping tool via python that extracts individual product page network responses from requests that return 403 errors. i started using selenium wire and i got it to work, but the main issue is the memory increasing more and more the longer i run it.

i’ve tried everything in order for it to not increase in memory usage, but ive had no success with it.

i’m wondering if anyone has had this problem and found a solution to access these requests without memory increasing over time. or if anyone has found another solution.

i’ve tried playwright and seleniumbase, but i didn’t have success with those.

thank you.

i’m running a kohls website scraping tool via python that extracts individual product page network responses from requests that return 403 errors. i started using selenium wire and i got it to work, but the main issue is the memory increasing more and more the longer i run it.

i’ve tried everything in order for it to not increase in memory usage, but ive had no success with it.

i’m wondering if anyone has had this problem and found a solution to access these requests without memory increasing over time. or if anyone has found another solution.

i’ve tried playwright and seleniumbase, but i didn’t have success with those.

thank you.