Job Details

Job Description

An employer in San Francisco, CA is seeking a technical Data Engineer to perform data migration from Splunk to Snowflake. Pull a subset of aggregated web server log data out of Splunk into Salesforce's Snowflake environment.

Benefits

• Allows us to understand when search engines & LLM's crawl our content, and which content they crawl.

• This data is valuable to the SEO team (to understand how frequently our content is being consumed by LLM's and search engines), to the content team (as part of the CPS score).

• Allows us to understand the number of 301 redirects and 404 error pages encountered by users accessing our site.

• This data is valuable to understand website health and to measure the impact of an internal project that is aimed to reduce the number of 301's and 404's.

• Data retention

• Splunk only keeps data for 3 months.

Milestones

• Build out the query pattern in Splunk to extract the data that's needed to answer the aforementioned points (we don't want to bring in everything).

• Store the data in an intermediate storage area (such as an AWS S3 bucket). Doing this allows us to be flexible in the final destination of this data. Today the final destination is - Snowflake, but in case one year down the line we decide to do everything in Data 360, this step will still be relevant).

• Build an ETL process to ingest this data into Snowflake.

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to ...@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy:

Skills and Requirements

• 5+ years ETL experience from Splunk to Snowflake Datacloud