The Office of Compliance Inspections and Examinations (OCIE)

At mid-decade, the National Exam Program joined industry-wide efforts to tackle the new problem of cybersecurity and reconsidered risk by bringing early-decade initiatives together into a new Office of Risk and Strategy. Throughout, OCIE upgraded technological systems with the ultimate goal of having better information than the regulated industry.

Cybersecurity

While it had been of increasing concern for years, the 2013 Target Department Store data breach, in which some 40 million customer account, credit, and debit records were stolen, may have marked the point at which cybersecurity rose from a luxury to a necessity. The OCIE response was the National Cybersecurity Exam Program.

Jane Jarcho, now at the home office, helped launch the initiative. Like other OCIE programs, the effort was intended to learn about the industry and to share knowledge widely. Unlike others, there was no industry resistance to this SEC push for compliance. The effort was undertaken, said Jarcho, “in partnership with the industry to help improve the industry for everyone's benefit.”⁽⁶¹⁾

By the time Regulation Systems Compliance and Integrity took effect in November 2015, special examination teams included technologists, cybersecurity experts, and information technology experts. The SEC’s program took another step up late in the decade with the assembly of a team of experts, both internal and external, to be deployed in case of a cyber event. These efforts culminated in the Cybersecurity and Resiliency Observations Report published in 2020.

Reconsidering Risk

While SEC examiners had long considered risk factors in identifying targets, Carlo di Florio brought a more holistic view of the subject to the National Exam Program. His conception of “enterprise risk management” held that in any given firm, risk categories are interdependent, thus concentrating the hazard. Instead of looking at multiple individual risks, therefore, examiners needed to better understand specific business models and the risks inherent in them.⁽⁶²⁾

But it took a large number of data points to evaluate risk in a business model. So in 2011, a new Early Risk Assessment and Surveillance Unit, led by longtime OCIE staff member Jim Reese, began identifying these types of data points and compiling registrant risk profiles. In 2013, the unit developed risk assessment programs for the exchanges and FINRA. Later, Reese’s unit introduced machine learning and text analytics, enabling examiners to search investment adviser ADV Forms for problematic keywords like “guaranteed,” and “risk-free.”

Since other OCIE teams were also developing increasingly sophisticated risk evaluation technology, in 2017 Pete Driscoll consolidated all of the work into a single Office of Risk and Strategy. By then the “risk team” was evaluating about 50 distinct factors in determining which firms to examine.

Technology

Every step of the OCIE transformation was accompanied by technological innovation, with the period of greatest change, by one estimate, occurring during Drew Bowden’s directorship.⁽⁶³⁾ Previously examiners had evaluated voluminous records in one of two ways. Sometimes OCIE personnel put trade blotter information into a spreadsheet, after which a particularly experienced staffer spent days searching the spreadsheet for patterns. Other times staff employed sampling, analyzing portions of data taken from an otherwise unmanageable amount of material. In either case, fraudsters stood a good chance of going undetected. OCIE had to be able to effectively search large, full collections of data.

Much of the data in question came from an exam platform with a long history. The National Exam Analytics Tool (NEAT) started at the Chicago Regional Office as a spreadsheet with trade blotter information. The Chicago team migrated the information to a more robust off-the-shelf database which was itself incorporated into a much more powerful program developed by OCIE quantitative analysts and the SEC Office of Information Technology.

The old spreadsheet could hold only a few weeks’ worth of data. NEAT, introduced around 2013, allowed quantitative analysts to run about fifty different tests on full data sets to detect behavior such as front-running and cherry-picking. Bowden warned potential violators that “we’re coming in to look at your data, and we’re not sampling.”⁽⁶⁴⁾ By 2018, NEAT was fully developed. “For once,” Said Driscoll, “the government had a tool that was better than industry.”⁽⁶⁵⁾

A program called TRENDS also originated in the Chicago Regional Office. As late as 2008, recordkeeping in the examination program still revolved around notepads, folders, and boxes. At about that time, each region began developing its own tools for digitizing work product. Eventually the Chicago version, dubbed the Tracking and Reporting National Documentation System, was universally adopted.

By the mid-2010s, OCIE’s technological center of gravity shifted from Chicago to New York and the Quantitative Analytics Unit led by Elcin Yildirim. In addition to transforming regulator-created databases into powerful digital tools, the Quantitative Analytics Unit also built customized data analytics tools and worked with the Office of Risk Assessment and Surveillance on system architecture and software tools designed to improve collection and analysis of registrant data. By the end of the decade, quantitative analysts became, in some respects, more important even than examiners. “I would choose a quant over five examiners at any point,” said Driscoll, “because the ability of the quant to analyze information across hundreds of firms gave us such a leg up.”⁽⁶⁶⁾