Skip to content

Experience

Apple, Inc.

Date of Employment: June 2016 - Present (6 years 9 months)

Type of Employment: FTE

Title: Site Reliability Engineer / Application Infrastructure Engineer

Facebook, Inc.

Date of Employment: February 2012 - March 2016 (4 years 2 months)

Type of Employment: Contractor (first 2 years) and FTE (last 2+ years)

Title: System Engineer

  • 2+ years of experience working directly with deploying IPv6 in a large scale Linux environment.
  • 4+ years of experience as a founding member of the IT Chef team.
    • Built IT Infra's multi-regional Chef infrastructure stack starting with the original implementation.
    • Lead engineer writing, testing, and implementing of the IT Chef code base.
    • Cookbook creation included basic system configurations to complex system automation workflows.
    • Orchestrated 600+ Chef code pushes involving cookbooks, roles, environments, and databags.
    • Authored Chef code to manage Chef itself to include managing its chef-clients on Chef nodes and auto-deploying Chef servers with Chef.
    • Authored an original password randomnizer in pure Ruby and integrated it with Chef seamlessly to securely randomize all root user passwords.
    • Authored a specialized piece of Chef automation code that dynamically dual-stacked RHEL and Ubuntu nodes with IPv6 network interfaces.
    • Authored all code to convert Chef cookbook based secrets to use encrypted data bags and migrated all Chef Linux nodes to support encrypted data bags.
    • Fully implemented a deployment of taste-tester, a Facebook specific Chef integration testing tool, on the IT Chef stack.
  • Self taught the Ruby Programming language both as a declarative DSL within Chef and as a general OOP higher language.
  • 4+ years of experience managing thousands of heterogeneous IT Linux systems (CentOS, RHEL, OEL, and Ubuntu) with Chef automation.
  • 4+ years of hardening Chef managed Linux systems with things including: LDAP auth, DUO 2-factor auth, OpenSSH security tuning, and automated sudo policies.
  • 4+ years of experience building and operating the multi-regional package management infrastructure for different Linux distributions.
    • Originally designed, built and implemented a multi-regional Linux driven HTTP heterogeneous package management service using Chef.
    • Authored an original package repository synchronization tool in pure Ruby that automatically mirrored public repos to keep them up-to-date.
    • Maintained and built all custom IT Infra packages that included RPM and DEB package artifacts.
    • Created a lightweight BASH package build script framework to assist in compiling source code based packages.
    • Mastered the usage of FPM, a ruby meta-packaging tool, that helped build custom code based packages.
    • Worked closely with Security teams to coordinate, push and mitigate packages affected by major security vulnerabilities.
    • Ensured all and any package spec files or custom build scripts were version controlled within git repositories.
  • 2 years of experience building, re-designing, re-building, and operating multi-regional global Facebook IRC service.
    • Rebuilt and migrated the entire Facebook IRC Infra to the IT Infra stack, including different backend systems, load balancers, and networking.
    • Designed and implemented the re-designed IRC infrastructure entirely in and with Chef so that it was completely codified infrastructure.
    • Configured the IRC implementation to be a multi-faceted Internet facing service.
    • Iterated upon the IRC infrastructure by upgrading the whole stack to work with IPv6 making it the first Internet facing IPv6 IT service.
    • Coordinated and communicated IRC migrations, outages, and upgrades to the company.
    • Managed monitoring IRC service health, users, and traffic by failing regional traffic over during incidents and upgrades.
  • 4+ years of experience in a fast paced, iterative development and operations environment.
  • 3+ years of experience operating and managing load balancer traffic infrastructure for IT Infra services, focusing in Chef, Package Management and IRC services.
  • 4+ years of experience provisioning virtualized Linux machines.
  • 4+ years of experience with on-call rotations and on-demand repair of large scale service outages.
  • 4+ years of experience with site/service event and incident response, incident root cause analysis, incident report creation, and incident review post mortem meetings.
  • 4+ years of experience working in line with Technical Project Managers to push and deliver innovative new services.
  • 4+ years of experience working with internal and external auditors to ensure compliance of system configurations for financial based infrastructure.
  • 4+ years of experience rebuilding entire infrastructure stacks to migrate and upgrade them to different datacenters locations.
  • 2+ years of experience actively participating in Disaster Recovery drills on core infrastructure services to test their resiliency and availability.
  • Participated in complete Infrastructure Destruction and Security Breach simulations. Also, know as Red Team exercises.

Google, Inc.

Date of Employment: September 2010 - January 2012 (1 year 5 months)

Type of Employment: FTE

Title: Unix Systems Administrator

  • Managed the engineering Ubuntu Linux development systems with BASH scripting, git code pushes, security package updates, and network monitoring.
  • Converted the Slide Design Team to corporate Google hardware and software.
  • Implemented a multiplatform SSHFS software deployment for Slide Teams to securely access remote file systems.
  • Converted all Slide Team members to use SSH key based authentication when accessing a bastion hosts.
  • Created a scripted account creation automation and management process for newly hired, exited, and existing Slide AU employees.
  • Improved Slide AU account setup with multiple BASH scripts for LDAP auditing, developmental playground initialization and conversion, file encryption, and Slide AU specific OS configurations installations.
  • Authored procedural documentation on Slide AU user account management, developer playground creation, SSH usage and ethics guide, new hires, terminations.
  • Assisted the Slide AU Systems Operations team with a major datacenter colocation migration and setup.
  • Supported multiple remote teams in China and Europe.

Slide, Inc.

Date of Employment: July 2009 - August 2010 (1 year 2 months)

Type of Employment: Contractor/FTE

Title: Systems Operations Admin and Facilities Assistant

  • Worked on the Systems Operations team with an emphasis on Linux systems administration.
  • Maintained, operated, and repaired corporate Ubuntu Linux servers and services.
  • Deployed FOSWiki, FreeGhost/FOG PXE imaging system, and Apple Netboot System Image Server to improve desktop and laptop distribution.
  • Administered the Mailman mailing lists, TWiki, and Kwok Information internal servers and services.
  • Supported 100+ local employees and multiple overseas offices using mixed operating systems environments.
  • Worked with Slide Facilities to help maintain, organize, and improve the three company office building floors.

Slide, Inc.

Date of Employment: July 2007 - September 2007 (3 months)

Type of Employment: Internship

Title: System Operations Intern

  • Collaborated with Systems Operations team members to support all Slide employee desktop and laptop computers.
  • Assisted in the physical hardware and software configuration maintenance of corporate Linux servers.
  • Collaborated with network engineering to fix and setup new corporate network infrastructure.
  • Implemented a new company wiki with TWiki to help improve employee communication, productivity, and sharing of project critical information.

Cal Poly Pomona University Library

Date of Employment: October 2005 - June 2007 (1 year 9 months)

Type of Employment: On-campus Student

Title: Student Systems Operations Admin

  • Supported all library employees from six internal departments.
  • Effectively maintained and ensured the functional and operational integrity of all computer labs consisting of 100+ computers of mixed operating systems.
  • Completed tasks to include resolving support tickets in an efficient manner.
  • Solved daily on-demand desktop software and user support problems for library employees.