Description
On behalf of Ivix, SD Solutions is looking for a talented Backend Developer (Crawler) to join a talented team.
SD Solutions is a staffing company operating globally. Contact us to get more details about the benefits we offer.
Responsibilities:
Conduct research and extract data using advanced web crawling technologies and techniques.Utilize Python libraries like Scrapy, Beautiful Soup, Selenium, or Playwright for web crawling and data extraction.Parse HTML and DOM structures using tools such as BeautifulSoup or lxml, leveraging XPath and CSS selectors for precise element extraction.Handle dynamic content from JavaScript-heavy websites using tools like Selenium or Playwright.Work with structured data formats like JSON, XML, and CSV to ensure accurate data representation.
Requirements:
Strong proficiency in Python programming language.Familiarity with HTTP/HTTPS protocols, headers, status codes, and HTTP methods like GET and POST.Experience in using Python libraries for web crawling, request handling, and data parsing (e.g., requests, httpx).Understanding of threading, multiprocessing, and asynchronous programming for handling concurrency in web scraping.Knowledge of unit and integration testing frameworks like pytest or unittest.Experience using developer tools like Postman for API testing and debugging.
Advantages:
Hands-on experience with real-time web networking and HTTP crawling.Expertise in databases or cloud storage services for managing extracted data.Proficiency in debugging Python code and optimizing workflows for scalability.
About the company:
Powered by artificial intelligence and machine learning, IVIX gathers and enriches publicly available business activity data to accurately identify businesses, their revenue, and the taxpayer entity.
By applying for this position, you agree to the terms outlined in our Privacy Policy. Please take a moment to review our Privacy Policy https://sd-solutions.breezy.hr/privacy-notice, and make sure you understand its contents. If you have any questions or concerns regarding our Privacy Policy, please feel free to contact us.
