Big Data Software Architect (Hadoop / Microsoft HPC ) (13259) Newark, California
We are building our own distribution and aggregation framework and want your fresh perspectives and exceptional engineering skillset implementing distributed computational systems!
As a senior engineering member of our Distribution & Aggregation group in Analytical Development, you will be a core contributor to the team building our distribution and aggregation computational framework. You will leverage your significant prior experience in building such systems to contribute to the design, implementation and deployment of our own distribution & aggregation grid. You will be responsible for large portions of the stack including integration of our analytical engines and many of the necessary infrastructure components needed for a robust, scalable, easy-to-maintain and fast processing framework. You will be challenged with the need to continually improve and optimize the system as job processing load increases and our clients demand faster response times for larger volumes of data. You will have the opportunity to truly make a difference as we innovate from the ground up in building RMS(one), our next generation product solution for the cloud. We want you to bring your ideas and software engineering skills to our dynamic, growing team!
We are looking for a great programmer who is motivated, smart and enthusiastic about building a distribution grid from scratch. You will be right at home in this position if you are someone who is not content with just writing code, but rather wants to understand how and why things “work under the hood”. An ideal candidate is curious by nature and thrives on digging into what is happening at the O/S level and understanding how the compiler interprets code. Your prior engineering experience may have included building a similar system or working extensively with Hadoop, Microsoft HPC or the equivalent. Excellent communication will be one of your strong traits. If you thrive on taking responsibility for delivering new solutions for a growing, dynamic product, we want to hear from you.
Required experience and skills:
- Superior programming skills in C++ and C#/.NET on a large code base. Expertise in another OO programming language can be substituted if prior experience is substantial and candidate has strong knowledge of C++ and/or C#/.NET
- At least one previous experience having designed and implemented a distributed, scalable computational framework such as a cluster-based “grid” or a high-throughput/low-latency parallel computing farm. Significant prior work with Hadoop/Microsoft HPC/similar cluster frameworks will be considered. Deep knowledge of the issues involved (e.g. performance, scalability, availability, concurrency, queuing, etc.)
- Superior knowledge of measuring performance, identifying bottlenecks (code and/or architectural design), and monitoring resource usage (memory, CPU, disk IO, network IO)
- Strong knowledge of communication protocols (TCP/IP, sockets, etc) and networking challenges in highly parallel, distributed systems; ability to design, implement and debug custom communication protocols
- 10+ years of programming experience; proven track record of on-time delivery of your work
- Excellent communication skills and proven ability to convey complex ideas to others in a concise and clear manner
- Strong software engineering discipline: you document, measure, test and tune what you build
- B.S. / M.S. in Computer Science or related discipline; Master’s degree preferred
Desirable experience and skills:
- Strong mathematical skills in algorithmic computations, statistics and probability
- Strong knowledge of a major RDBMS platform (e.g. Microsoft SQL Server, Oracle), data modeling and data access and sharing patterns in distributed systems
Founded in 1989 as a spinoff of research done at Stanford University, today RMS is the world leader in catastrophe risk modeling. From earthquakes, hurricanes, and floods to terrorism, longevity, and infectious diseases, our models provide critical risk management solutions to financial institutions around the world. Based in our Silicon Valley headquarters and with offices throughout North America, Europe, and Asia, our team of more than 950 professionals may be larger than it used to be, but we"re still as ambitious and innovative as ever. Whether quantifying risks due to climate change, simulating how medical breakthroughs could extend life expectancy, delivering supercomputing in the cloud, or transforming an industry with our next generation technology platform, our passion is innovation driven by the courage to tackle problems others reject as unsolvable. At RMS, we believe the result of deeper knowledge is positive change. Our ultimate goal is to create a more sustainable and resilient world through a better understanding of catastrophic events. To find out more, visit www.rms.com. RMS is proud to be an equal opportunity employer.