One of the biggest pain points Cloud Service Providers (CSPs) face when implementing FedRAMP is around change control, required by CM-2, CM-3, CM-4 and CM-5. Cloud services change rapidly, with hyperscale cloud environments introducing dozens or even hundreds of changes per day. Per FedRAMP’s CSP Significant Change Policies and Procedures, changes deemed “significant” must be approved by the Authorizing Official (AO), but what’s considered significant is often a gray area. Manually reviewing every system change is untenable at the scale of cloud – it simply breaks the cloud business model. On the other hand, introducing changes also introduces risk – changes can introduce misconfigured software, risky supply chain vendors, or vulnerabilities. It makes sense that government agencies want visibility into changes. It also makes sense that CSPs want to use cutting edge, automated DevSecOps practices such as continuous integration and continuous delivery/deployment (CI/CD) for their Public Sector focused offerings. Implementing an effective, holistic change control process is the key to ongoing success in the Continuous Monitoring phase of a FedRAMP ATO. Below is an approach that finds the sweet spot and allows changes to occur rapidly at scale while also providing visibility into risk.
Per NIST SP 800-128, Guide for Security-Focused Configuration Management of Information Systems, the scope of change control includes:
- Regular patching, maintenance, disposal, and personnel changes for the system
- Introducing new technologies, product features, processes, or customer requirements
- Responding to Government priorities such as OMB Memos, DHS Binding Operational Directives, or NIST revisions to SP 800-53
The changes above may or may not be considered significant, but understanding the scope is important. All of these types of changes must be considered and included in the overall change control process. Many companies already have change control policies, but they are organization-based and not dedicated to the specific Cloud Service Offering. The following actions outline a system-level approach to change control that incorporates and consolidates processes across engineering teams, provides visibility, establishes governance, improves documentation accuracy, and iterates changes more rapidly.
Achieve FedRAMP Compliance for your cloud service offering. Get in touch with 38North Security today.
Action #1: Establish a System-Level Governance Structure
I’ve worked with dozens of CSPs, and teams are usually siloed by organizational function rather than falling under a single System Owner. SecOps teams, Product teams, Engineering teams, Development teams all fall under different Directors and VPs who may also be responsible for an entire portfolio of products, some of which may not require FedRAMP compliance. FedRAMP program management is usually placed under the Security and/or Compliance team. Executive oversight varies wildly. I’ve seen leaders from Product, Engineering, Security, or Legal arbitrarily designated as the FedRAMP “System Owner” who signs off on system risk on behalf of the organization, sometimes without the direct authority to enforce system-level policies. 38North recommends that organizations establish the following roles to create a FedRAMP system-specific governance structure:
- Establish an Information System Security Officer (ISSO) per PL-2 – A senior practitioner level security role who is directly responsible for ensuring system documents are updated, Continuous Monitoring and Annual Assessment submissions are delivered and reported to the AO, Significant Changes are submitted, and Plans of Action and Milestones (POA&Ms) are remediated on time.
- Optional: Establish an Information System Security Manager (ISSM) – A Senior Management level security role who is responsible for oversight of the FedRAMP Program, and potentially other U.S. Public Sector focused programs (StateRAMP, TX-RAMP, CMMC, DoD SRG). This role is optional for smaller systems, but critical for large organizations who are managing complex environments and/or multiple authorizations.
- Establish a System Owner – An Executive-level role who is responsible for formally approving system documentation, including significant changes. This person should have enough authority to be accountable for system risk on behalf of the organization. The person should also have a solid understanding of information security, though they don’t necessarily have to be in a dedicated security role in the organization.
- Establish system-specific key roles – There are certain roles that are required for FedRAMP Moderate and above, but are recommended for all baselines. These roles are dedicated to performing tasks around control implementation. These include:
- Privacy representative per CM-3(4). Typically, this is a person with a legal background who understands privacy risk and has a certification such as the CIPP.
- Supply Chain Risk Management Lead per SR-2(1)
- Contingency Planning Director (CPD) per CP-2 and defined in the Information System Contingency Plan Template
- Incident Response Lead (IR-8)
- Optional but recommended: Continuous Monitoring Lead. This role is responsible for ongoing vulnerability management and reporting via monthly Continuous Monitoring deliverables to the AO.
- If responsible for implementing FedRAMP controls, then representatives from Human Resources, Legal, Product, Engineering, Development, Customer Success, Sales, and others must be identified.
The functional management structure may be organizationally-defined, but there should be a direct line of communication from the system-specific key roles to the ISSO, the ISSO to the ISSM, and the ISSM to the System Owner.
Action #2: Create Two Change Control Groups
Change control should be divided into two gates: technical review and executive oversight.
- The Technical Working Group (TWG) serves as the practitioner-level review. The ISSO should serve as the chair (per SA-05d). Practitioner-level stakeholders representing all the system-specific key roles identified above ought to be included. The TWG is responsible for identifying and reviewing potentially significant changes and coordinating realistic plans and timelines for recommendation.
- The Steering Committee serves as the executive oversight. The ISSM is appropriate to serve as chair, with the ISSO and Privacy representatives present. Membership should include the System Owner, as well as the senior leadership of each organizational function with a key role responsible for implementing FedRAMP controls. Others may be included as needed. The Steering Committee is responsible for formally approving Significant Change Requests (SCRs) and approving policies/procedures/System Security Plans annually (pending AO approval). The steering committee should also ensure that adequate resources are dedicated to accomplishing significant changes within the timelines communicated to the AO.
These two functions should not be combined. Regarding the logistics of when and how to meet, two approaches may be taken. The more manual approach is scheduled recurring meetings. The cadence may be determined based on the number of changes that need to be reviewed and approved. Typically, the TWG ought to meet at least weekly, whereas the steering committee may meet less often, but may need to have ad hoc meetings to expedite specific approvals. An alternative approach is to bake in required discussion and approval within an automated version control or ticketing workflow, thus avoiding the need for regular meetings (and inevitable meeting fatigue or scheduling issues). This approach lends itself well to an automated CI/CD process, which is described in more detail below.
Learn more: Here Are Five Reasons You Won’t Achieve FedRAMP Authorization
Action #3: Create a Change Control Process Workflow
Achieving speed at scale depends on an efficient process. The Security Impact Analysis (SIA) is a list of questions that distinguishes between a “regular/maintenance” change versus a security or privacy impacting change. Changes that are security or privacy impacting are likely to be considered significant. NIST SP 800-128 provides an appendix with a SIA template. This template is a great starting point for crafting an organizational SIA for FedRAMP change control. This “Yes/No” questionnaire may be baked into an automated check via software version control tooling (GitHub, GitLab) or incorporated into a ticketing system such as JIRA or ServiceNow. Here are some recommended questions to determine whether a change is significant:
- New Technology (New OS variant, including COTS and Appliances)
- Use of new external services (e.g. ticketing system, monitoring system) in support of the cloud service
- New Cloud Service Offering or Feature
- Removal of system components or service offering
- FIPS 199 Categorization Change (e.g. Moderate to High)
- PaaS or SaaS changing IaaS Provider
- Adding/Removing security controls
- New cryptographic modules or services or changes to existing modules/services
- Changing alternative (or compensating) security control(s)
- New data center or moving to new facility
- New interconnection or changes to existing interconnection
- Scanning tool changes
- Major upgrade of OS
- New system monitoring capabilities or replacement of system monitoring capabilities
- New Virtual Server(s)
- New/upgrade of DBMS (MS SQL, Oracle, etc.)
- New Major Code Release
- New authentication mechanisms or changes to existing mechanisms
- New boundary protection mechanisms or changes to existing mechanisms; changes to routing rules
- Change in cloud service ownership that would result in major changes (e.g. change to contingency planning or incident response processes/capabilities)
- Changed or updated backup mechanisms and processes
- Movement of information system data to a different system boundary
- Introduction or generation of Privacy impacting data (PII, PHI)
If the answer to all of these questions is No, the change ought to be considered a standard change, and may proceed per the usual deployment process. If any of these questions is Yes, then the change is brought to the Technical Working Group for review. This step ensures that the TWG is not bombarded with standard changes that occur frequently in cloud environments. It also is a short and sweet list that engineers can complete as part of a service ticket whenever they make a change, only taking up a few additional minutes of their (very expensive) time. Much of the questionnaire process may even be scripted based on data within the change request itself. For more detail around how CI/CD pipelines may be architected, refer to NIST SP 800-204D Strategies for the Integration of Software Supply Chain Security in DevSecOps CI/CD Pipelines and NIST SP 800-218 Secure Software Development Framework (SSDF) .
If the TWG determines that a change is indeed significant, it moves to the Steering Committee for internal approval. Additional items to discuss may include changes introduced by other stakeholders. Once internally approved, the ISSO submits the SCR to the FedRAMP PMO and the AO. If necessary, a 3PAO tests and submits validation as well. Once the AO approves, the change may be implemented in production, and the ISSO ensures that the change is reflected in the system documentation. Below is a visualization of the recommended Change Control process workflow that can be successfully implemented at scale.
Partner with 38North Security to dive deeper into implementing Change Control. Let 38North Security empower your organization to maintain the speed and scale required to maintain a state-of-the-art Cloud Service Offering, while also managing risk around system changes appropriately.
