Responsible for one of the top destinations at Yahoo! (Yahoo! OMG). This includes all aspects of the site, and meeting existing SLA standards for both site performance and availability. At peak, omg.yahoo.com would get in excess of 7k requests a second, which would put significant strain on the database back end, the API, and multiple caching layers.

Responsible as the primary engineer for numerous second-tier properties such as Yahoo! Starwars, Yahoo! Entertainment, and Yahoo! Shine. This included working with development engineers and managers to analyze site performance before major releases, and for troubleshooting issues related to SLA misses.

Launched Yahoo! Starwars (Worlds)- This included all aspects of project management, including obtaining hardware by justification of site performance expectations, as well as working with many groups (site ops, net ops, etc.) to ensure goals are met based on established project plan. Later, worked with the transition of the site's maintenance to our counterparts in Bangalore India

Was responsible for doing a detailed analysis and reporting on the feasibility of implementing a massive Squid caching layer in front of the entire storage farm to improve performance, and reduce load on the back-end. Work included doing a size distribution of 10's of millions of cachable objects, as well as their request frequency. After this, determining number of required systems to achieve a significant level of performance, and creating a roll out plan for bringing it live.

Implemented a multi-tiered Nagios installation using a completely custom roles based administration tool (I wrote it in Perl) to manage 2.5k nodes, and 25k active service checks. The design allowed for dynamic slave allocation based on increased usage, and was responsible for finding limits with Nagios 2.6's design, and worked with Ethan (Nagios creator) to improve performance.

Was the primary administrator for their 4 PB multi-tiered "HSM" NAS storage environment that was receiving 2.5TB of data a day. Work included debugging file corruption issues, and NIS issues with mounting the filesystems.

Leading a two person team to test IBM's SVC (SAN Volume Controller) for possible integration into our existing SAN environment to provide a layer of abstraction for over 200TB of storage. The SVC project will also include using Veritas' VVR (Veritas Volume Replicator) to allow for replication of data to other datacenters, and for DR planning.

A part of a small team that is responsible for the daily maintenance of all production servers (mostly all UNIX). This includes the website, database backend, SAN environment, and systems to support the nationwide distribution hubs. Through performance monitoring, analysis, troubleshooting, and mitigation, we obtain a very high level of performance and reliability.

Improved Netflix's kickstart solution with a PHP-based solution (which I wrote) that allows for very rapid deployment of Linux servers. The new system allows the delegation of system building to less skilled admins, and improved the build time from 1 hour to about 8 minutes.

Did basic SAN storage allocation for the expansion of database filesystems. This included assigning LUNs to the system, and expanding filesystem using VxFS and VxVM. Storage allocation was done using both the command line and the ESS GUI for IBM Shark and FAStT storage.

A part of the migration team to move from the current bladecenter deployment to virtual servers leveraging IBM OpenPOWER P720's and Linux.

Responsible for the 24/7 upkeep of the entire Solaris/Linux production environment. This includes all DMZ hosts (web servers, smtp gateways, etc.) as well as the lab farm, and internal core corporate infrastructure Unix systems. Total systems under my responsibility were greater than 200.

Was the architect and developer of a Perl based inline hybrid HSM application that abstracts multiple storage devices (in this case, Network Appliances), and provides version control for tens of terabytes of data. Features included:

• Capable of directory-level locking during check-out/check-in. This allows other operations to be performed in parallel that do not threaten data/file integrity.
  
• Wrote the interface as a Layer-4 protocol so that it can easily communicate with the numerous applications already in house.
  
• User account management, and access restriction.
  
• Ability to merge multiple Network Appliances into a single virtual directory to provide a uniform file structure to the users.
  
• User aware in that during file check-in arbitration, the user is able to see who previously checked out/in a file to help determine if their changes can be overridden, or merged back into the repository safely.
  

Was the architect and developer of a corporate-wide spam solution where none existed prior. I used a Markovian based spam algorithm as well as custom Perl scripts that operated on the mail gateway to munge the spam message as an attachment to prevent malicious code from running during preview. All the original headers are left intact so the user can still filter the message as they wish. Learning is all email based- sending corrections back to the mail gateway so it can learn from its errors.

Responsible for the identifying of high-depreciation Sun systems, and building a migration plan to move those systems to cheaper commodity based hardware and Linux.

Due to a severely reduced IT staff, constantly finding ways to simplify complex systems to help us in managing those environments. Areas included how Nuance publishes internal DNS records, user account management, LDAP maintenance, etc.

Installed a PHP based Linux kickstart solution that allows for the building of profiles based on IP address- this system allows for rapid installation of new Linux systems simply by defining profiles/macros for those systems. I wrote this application prior to Nuance, and can be found at: http://freshmeat.net/projects/phpkickstarter/

Responsible for the 24/7 upkeep of 60+ production co-located servers, and 100's of production in-house servers. SLA agreements required that the production servers have at most 30 minutes a month of downtime for maintenance (99.932% uptime)- this goal was easily met.

Was the architect and project manager for the trans-data center relocation of our two external websites and all equipment. Responsible for planning the relocation and outage of the entire lifecycle from scoping, to development and testing, to completion. This project had extremely high visibility amongst senior management and was the highest project to be completed by the company. Accomplishments included:

• Achieved an "outperform" rating from the Director of IT by bringing everything online in less than 24 hours- the target goal was 48 hours.
  
• Was the coordinator between different departments including networking, facilities, and application delivery to ensure that project deadlines were met, and dependencies were resolved.
  
• Planned the data center floor plan layout- considered power/HVAC requirements to ensure reliable service, and efficient rack layout (given limited space) to allow maximum growth to reduce the risk of moving equipment in the future.
  
• Achieved no downtime (some decreased functionality though) by forcing a split brain in the HA environment, applications affected included Apache, Oracle, and Weblogic.

Was the project manager and architect for the building out of a network and it's systems for the support of Incyte's new products. This required project planning spanning multiple departments (engineering, marketing, and sales) to determine a best-fit solution by keeping costs down and allowing for growth based upon expected growth forecasts. Responsibilities included deciding what platform and hardware to use, user management, and security risk mitigation.

Was the architect and builder of the backup system for one data center to allow it to handle 100's of Gigabytes with a short backup schedule. Due to strict time constraints, numerous techniques were required to reduce the window of vulnerability during upgrades to the data being provided for the online customers.

Was the architect and builder of their host monitoring system to provide an efficient system to monitor 2000+ nodes in multiple networks. This included writing custom Perl modules to extend functionality, and creating packages to aid in duplication of the custom configuration.

Was the architect for a centralized email processing system to improve efficiency of analyzing emails sent by the 2000+ production systems. Due to the variety of emails received, this helped reduce overlooking key important notifications, and reduced the amount of time spent perusing the emails significantly.

Was the architect and builder of a RedHat Linux kickstart server to aid in building new Linux cluster nodes. I leveraged the PXE standard, and dynamic DNS (found in Bind 9, and ISC DHCP 3.x) to allow systems to be rebuilt just from the network card (not needing a CD or a floppy). This allowed for rapid build out of new machines in anticipation of increased cluster processing needs.

Was the architect and builder of a Linux cluster using Platform LSF to assist in processing/analyzing multi-terabyte data volumes to be used for the online product.

Was a key member in a small security taskforce setup to assess and mitigate security risks for a new product expected to generate roughly half of the company's revenue. I am responsible for the online version of the product.

Was the sole architect and systems administrator for the entire web-based presence including the corporate site and subscription site spanning three data centers. The subscription site that houses all of the web-based products is used heavily by many tier-1 pharmaceutical companies. Due to the high requirements of the customers, security, accountability, and QOS requirements are extremely high. The systems included OpenBSD, Solaris 2.6 to 8, and Linux for clustering needs. Due to the large number of systems supported, a high amount of automation was required to provide first-class service.

Supported a group of developers that worked in the above environments, and provided a high level of service for them. During my tenure, I received two awards for my services to the company and their group.

Responsible for the 24/7 upkeep of the entire Solaris/Linux/OpenBSD production environment. This includes all DMZ hosts (web servers, smtp gateways, bastion hosts, etc.) as well as internal core corporate infrastructure Unix systems. During my tenure, 40+ systems were under my responsibility.

During the potential sale of Riffage, a due diligence crew came and evaluated the production and corporate infrastructure that I designed and built. The president of Riffage told me that the crew rated my design as one of the top of about 20 companies that they reviewed over a two-month period.

Was the architect and developer of a fully automated file processing system that was capable of operating over a LAN or WAN, and also designed to be clusterable across heterogeneous Unix machines. This program operated continuously for over a year until a newer version replaced it (which I also wrote).

Was the project manager tasked to find an adequate data center co-location, including the management of all contracts and logistics. Was the systems architect that designed the network infrastructure, rack layout, and was the architect that determined specifications and forecasting what hardware would be required given current expectations. Evaluated countless vendors demonstrating cutting-edge technologies to determine a best-fit scenario for the company.

Was the systems architect that designed and implemented the corporate network and server environment (which had greater than 50 nodes and 15 servers), which included OpenBSD, Linux, and Solaris 2.6 to 8.

Designed and implemented the corporate mail system, and production Bind DNS servers (including satellite offices). The mail system included a specialized mail gateway that filtered both incoming and outgoing mail using a custom PCRE based filter essentially making the company immune to popular mail viruses such as the "Ilove you" virus and variants.

Designed an extensive auditing system using Tripwire, ingress and egress firewall rules, and snort reporting that aided in the identification of numerous attacks and compromised systems, which brought a swift response.

Was the systems architect that designed the security policies that controlled access in and out of the facility. This includes firewall design, access controls, and implementation of operating system policies to ensure a highly secure environment.

Designed a cost-effective way to obtain high-availability for the production environment leveraging custom scripts and taking advantage of inexpensive technologies to reduce costs.

Created and maintained corporate and production policies. Directly managed three subordinates. Area of authority included the multi-million dollar data center (that I worked on below), as well as the network for central HQ, and offices nationwide.

Improved and maintained an already existing heterogeneous Unix environment for a pre-production website- including security risk assessment and mitigation, creation of a reliable system, etc. Variants of Unix included Linux, OpenBSD, FreeBSD, and Solaris 2.6, 7, and 8.

Wrote a program called mp3_check that is playing a pivotal role in the quality of the music files available of the website. After being made publicly available, there have been over 8,000 downloads of the program, and I frequently get kudos from people around the world. The page for the program is located at freshmeat.net. People use it from satellite transmission processing, to extraction of MP3s from Shockwave Flash files.

Was the architect and developed all of Diamond Multimedia's Online Store, including the customer interface, database schema, transaction system, and the administration back-end.

Maintaining enterprise databases (Microsoft SQL Server, and Oracle), supporting developers in a Unix development environment under Solaris 2.5.1 and 2.6, and developing applications for online/Internet related purposes.

Created an enterprise level WAN based e-commerce data exchange system that provided a critical function in managing shipments of huge quantities of Rio MP3 Player product to third-party distribution centers nationwide.

Implemented a remote server administration system for data replication and clustering capabilities to Diamond's co-location at Digital Island in Hawaii for their international customers.

Created an online HTML page generation system used to build static pages for certain high-load areas of the production website.

Created an HTML based mass emailing system (subscription based) used by Diamond Multimedia for direct marketing.

Created intranet-based reporting tools for measuring e-commerce and direct marketing results of mailings and promotional specials on the online store.

Created an extranet for field engineers, allowing secure access to current product information, including features such as automatic email notifications of changes and threaded discussion forums.

Developed an FTP/HTTP staging server for web development, including a tiered security model, automatic notification of files checked-in for quality-assurance, automated data-replication of approved files to production servers.