So, you’ve developed a FedRAMP Risk and Authorization Management (FedRAMP) Authorization Package for your cloud service offering and submitted it. Congratulations! That’s quite an achievement. Give yourself a pat on the back.
Now, after you’ve finished your happy dance, you need to start preparing for your next big milestone in the process – a security assessment conducted by a Third-Party Assessment Organization (3PAO). Basically, an accredited third-party assessment organization assesses your information system and validates the information written in your System Security Plan (SSP) and associated attachments.
No need to get worried though, this post will tell you what to expect and how to prepare.
The 3PAO Assessment can be broken down into three parts:
- Manual Control Testing
- Compliance & Vulnerability Scanning, and
- Penetration Testing.
Let’s review each part separately.
Manual Control Testing
Manual control testing comprises the bulk of the assessment. You’ll interface with the 3PAO team who will be testing and validating the security controls. Manual control testing consists of three different assessment methods:
- Interview,
- Examine, and
- Test
During this phase, the 3PAO will collect artifacts used as evidence to show that control requirements are implemented correctly. When the 3PAO is unable to collect evidence, it typically results in a finding as they could not validate the control implementation.
Interview
The Interview phase consists of discussions between the 3PAO team and your personnel covering how the security controls are implemented. The 3PAO will coordinate with your team to schedule dedicated meetings to discuss each in scope control by control family. During these sessions, assessors will start out with general questions to ensure that what you say in interviews backs up what’s written in the associated implementation statement in the SSP.
Based on your answers, the questions will get more specific. The assessors aren’t trying to trick you. They are merely trying to gain context about how security controls are implemented and clarify how they can be tested.
Here are some sample interview questions that an assessor may ask for AC-2(2) – Account Management | Removal of Temporary / Emergency Accounts:
- Are temporary or emergency accounts used?
- Do you remove or disable temp/emergency accounts?
- What’s the process for removing/disabling temp or emergency accounts?
- Is there a time period defined for how long emergency and temp accounts can be used before they’re automatically disabled or removed?
If you’re unsure of who should be included for each interview session, ask the 3PAO for a list of the topics that will be discussed, if it hasn’t already been provided, so you can invite the knowledgeable personnel.
Examine
The Examine method consists of the 3PAO team requesting and reviewing documentation, such as the information security policies, associated procedures and plans, etc. As you know from assembling the FedRAMP authorization package, there are numerous associated documents that must be developed such as an Incident Response Plan, Access Control Policy, etc. The 3PAO will provide a list of policies, plans, and procedures that they will need to examine to ensure that the documentation meets the control requirements and contains adequate information.
This assessment method also pertains to examining supporting documentation, such as SLAs, access request forms and meetings minutes that were developed or created to support system processes. Ensure that all system documentation is properly dated with a review or revision history table to show that they have been reviewed according to the defined frequencies.
Some examples of artifacts/evidence that will be requested for examination include:
- Policies and procedures pertaining to all the control families
- System inventory, Contingency Plan, Incident Response Plan and Plans of Action & Milestones
- Audit log records, training records, maintenance records, etc.
- Service Level Agreements, Interconnection Security Agreements and vendor reviews
Test
The Test method consists of the 3PAO team testing the functionality of the cloud service offering to ensure that the FedRAMP control requirements are met, and the implemented security controls are configured correctly. This could take place via a screensharing session where a system administrator shares their screen or via in-person with the 3PAO observing by shoulder surfing.
The Test and Interview methods overlap a bit. During dedicated sessions, the assessor will start off with questions about how the security control is implemented, and then request artifacts/evidence to demonstrate that what was described during the interview portion is implemented. As an example, during a screen sharing session the 3PAO will request to take their own screenshots, or have you provide screenshots that will be used as artifacts.
If there is sensitive information that you do not want to be captured, please inform the 3PAO so they can write a statement describing what was displayed rather than capturing the sensitive information in a screenshot. Another alternative is providing a redacted screenshot to the 3PAO.
Below are some examples of artifacts/evidence that the 3PAO will collect as part of the Test method:
- Configuration Settings – The 3PAO will request evidence showing how the environment is configured to validate that the implementation is meeting the control requirements.
- Example: Record Time Stamps (AU-8/AU-8(1)) – The 3PAO would request configuration settings showing whether the components are configured to sync with FedRAMP approved NTP servers, and how often this sync is configured to occur.
- Functional Tests – The 3PAO will request that you walk through the process of meeting the control, to further validate that configuration settings are being enforced.
- Example: Unsuccessful Login Attempts (AC-7) – In addition to reviewing the configuration settings, the 3PAO would request the demonstrator to input incorrect passwords to validate that the account is locked after the defined number of consecutive invalid logon attempts.
- Tickets/Results – The 3PAO will request evidence of a process that is being followed or implemented.
- Example: Incident Reporting (IR-6) – The 3PAO would request evidence of tickets where incidents were identified and the incident response process that is detailed in the Incident Response Plan is followed.
Compliance & Vulnerability Scanning
The 3PAO works with the scanning team to conduct vulnerability scans of all assets within the authorization boundary. The scanning consists of network/OS scans, databases scans, and web application scans in accordance with RA-5. Compliance scans against the security configuration checklists utilized within the environment are also required, in accordance with CM-6.
These scans need to be authenticated using privileged credentials as required by FedRAMP. However, you do not need to provide any credentials to the 3PAO. The 3PAO will review the scanner settings, witness your team kicking off the scan, and request the raw results.
For initial assessments, you’ll need to ensure that all Critical and High scan findings are remediated and do not show up in the scan findings, or else you risk not receiving an authorization recommendation from the 3PAO.
Penetration Testing
The penetration test is considered part of the security control testing. However, you will most likely interact with a different team from your 3PAO, one that is solely focused on penetration testing.
The penetration test consists of a simulated cyberattack against the in-scope components and systems. This could include any web applications, mobile applications, networks, and physical site testing. Social engineering is also inscope for a FedRAMP penetration test.
You’ll work with the 3PAO penetration test team to develop a Rules of Engagement detailing what types of testing will occur, when they will occur, and what IP addresses/URLs will be in scope. During this phase, the penetration test team will collect screenshots related to their testing. These will be used as evidence in the report.
It is important that the 3PAO penetration test team has the necessary access, accounts, and other relevant information that they need to conduct the penetration test. An issue like not having the proper access can delay the test, which has the potential to delay the assessment.
Tips for a Less Painful Assessment
Now that we’ve gone over the basic parts of the security assessment, let’s go over some additional tips to make the assessment process as painless as possible.
Considering time is money and reviewing all the control families can take a significant amount of time, it’s best to ensure that the appropriate parties are available during the interview to ensure the assessment goes smoothly and avoid any delays. For example, whereas your system administrators may participate in discussions regarding the Access Control family, they most likely will not be needed for discussions regarding the Personnel Security family.
During the interview sessions, it’s recommended that the personnel answering questions are familiar with the NIST controls that they are responsible for, so they understand how to speak to meeting the requirements. It’s recommended to have the SSP handy during the sessions so it’s easier to follow along and paraphrase the written implementation statement.
As mentioned in a previous section, interviews typically follow-up with a request to provide evidence related to the interview statement provided. For example, if you state that your process includes submitting tickets to get X and Y accomplished, expect assessors to request samples of previous tickets to show that you’re following the process you described.
Also, do not guess at what types of evidence the 3PAO is looking to capture during the testing sessions. If you do not understand the request, feel free to request additional clarification. If you feel that their approach is off, let them know! At the end of the day, you are most knowledgeable about your cloud service offering.
Assessments can experience delays due to the appropriate personnel not being available or due to delays in providing the requested evidence. Hopefully now you have a better idea of what to expect during a 3PAO assessment and what will be requested of you and your team to mitigate any possible delays ahead of time.
