We are looking for a Data Engineer in Tehran, Alborz, Esfahan, Fars ,Khorasan-E-Razavi and Azarbayejan-E-Sharghi

Responsibilities	Qualifications	Subject
Create and maintain optimal data flow Set up/use different APIs to extract/transfer external data into data analytics platform (more likely will be Azure) Scrap large, complex data sets from multiple data sources (multiple websites) and integrate them Identify, design, and implement internal process to collect the words need to be scrapped and end user interface to enter the words Build the infrastructure required for optimal extraction, transformation, and loading of scrapped data from a wide variety of sources/websites using SQL and Azure ‘big data’ technologies Confident with scrapping required web data and loading required data into analytics platform Keep our data separated and secure through cloud based applications Create meta data catalogues/dictionaries from source to target and visualise data mapping to increase efficiency of team in finding and utilising data Work with data and analytics experts to strive for greater functionality in our data systems	Confident in scrapping web data Confident with Azure cloud environment: Azure factory, SQL server, Data Bricks Confident with using multiple APIs to extract data Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases Experience building and optimizing ‘big data’ data pipelines, architectures and data sets Strong programming skills (Python, Scala, or similar languages) Build processes supporting data transformation, data structures, metadata, dependency and workload management Strong social media data extraction/scrapping experience +3 years of experience in a Data Engineer role Experience with relational SQL and NoSQL database	Data Engineer