高阶16 - AWS上的合规自查神器-Prowler

我们所处的行业是医疗和生命，涉及到人类的方方面面，数据的合规性和隐私性是我们必须重视的问题，否则 就会成为愈来愈紧的法规下的反面教材，但是合规那么复杂，往往需要找一专门做合规的公司来咨询，费用不菲。 而对于日常使用者的我们，有什么工具可以帮助我们自查合规性，并提出整改建议呢？ -- D.C

废话不多说，先上图

什么是Prowler

Prowler is a command line tool for AWS Security Best Practices Assessment, Auditing, Hardening and Forensics Readiness Tool.

It follows guidelines of the CIS Amazon Web Services Foundations Benchmark (49 checks) and has more than 100 additional checks including related to GDPR, HIPAA, PCI-DSS, ISO-27001, FFIEC, SOC2 and others.

简单理解，prowler 是一款帮助我们自查合规新的命令行工具，有了它，我们就可以定期审视AWS上的资源，是否符合安全合规的最佳实践了。这也得益于AWS优秀的api接口，使得这种跨service的检查变得如此敏捷和自动化，至少目前我还没听说其他云商有这么简单的合规性检查工具的。

prowler支持180多种合规检查，这些检查都和以下的合规要求相关：

• Identity and Access Management [group1]
• Logging [group2]
• Monitoring [group3]
• Networking [group4]
• CIS Level 1 [cislevel1]
• CIS Level 2 [cislevel2]
• Extras see Extras section [extras]
• Forensics related group of checks [forensics-ready]
• GDPR [gdpr]
• HIPAA [hipaa]
• Trust Boundaries [trustboundaries]
• Secrets
• PCI-DSS
• ISO-27001
• Internet exposed resources
• EKS-CIS
• FFIEC
• SOC2
• ENS (Esquema Nacional de Seguridad of Spain)

它支持：

• 获得一个合规性报告（格式有：HTML, CSV, JUNIT, JSON or JSON ASFF）
• 可以把合规问题直接发送到 AWS Security Hub 国内已有
• 支持自定义合规性检查
• 并行/串行检查名下所有AWS账号
• 可以跑在本地机器，EC2，Fargate 或者其他容器里，以及AWS的Codebuild, CloudShell 和 Cloud9.

本文以安装在EC2为例。

准备工作

• 首先，起一台新的EC2实例，如果只是为了做合规检查用，那么开个t系列就可，定期检查完可以直接stop。

• 登录这台EC2，安装组件：

# install awscli, detect-secrets，ec2上默认已经装了awscli,如果是本地就是pip install awscli detect-secrets 
$ pip install detect-secrets

# install jq，跑prowler会调用
$ sudo apt install jq

# install prowler
$ git clone https://github.com/toniblyx/prowler
$ cd prowler

# 配置aws configure
$ aws configure
AWS Access Key ID []:xxxxxxxxxxxxx
AWS Secret Access Key []:xxxxxxxxxxxxxxxxx
Default region name []:cn-northwest-1
Default output format [json]:json

• 给这台EC2的IAM角色赋予它进行合规性检查的权限Policy，当然如果你的账号是admin权限，就可以跳过这一步了。

# 这两个是aws 自带的
arn:aws:iam::aws:policy/SecurityAudit
arn:aws:iam::aws:policy/job-function/ViewOnlyAccess

如果要检查 Extras 合规下的所有服务，还需要新建一个policy如下并把它附加给这台EC2的IAM Role。

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "dax:ListTables",
                "ds:ListAuthorizedApplications",
                "ds:DescribeRoles",
                "ec2:GetEbsEncryptionByDefault",
                "ecr:Describe*",
                "support:Describe*",
                "tag:GetTagKeys"
            ],
            "Resource": "*",
            "Effect": "Allow",
            "Sid": "AllowMoreReadForProwler"
        }
    ]
}

如果要让prowler和AWS Security Hub 集成，自动发送报告给AWS Security Hub, 就再添加如下policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "securityhub:BatchImportFindings",
                "securityhub:GetFindings"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

开始使用

• 直接运行 prowler 命令，他会使用你在aws configure里设置的信息（保存在~/.aws/credentials），去check整个region的安全合规性，如果credentials里有多个region，他会全部检查。

$ ./prowler

• 如果不想装那么多依赖，可以用Docker来运行：

$ docker run -ti --rm --name prowler --env AWS_ACCESS_KEY_ID --env AWS_SECRET_ACCESS_KEY --env AWS_SESSION_TOKEN toniblyx/prowler:latest

• 使用 -l 命令可以列出所有它会进行的合规性检查以及与之对应的合规group名称（如PCI，HIPAA）, 使用 -L 列出所有合规Groups, 使用 -l -g <groupname> 列出这个group内包含的内容。

$ ./prowler -l -g hipaa
                          _
  _ __  _ __ _____      _| | ___ _ __
 | '_ \| '__/ _ \ \ /\ / / |/ _ \ '__|
 | |_) | | | (_) \ V  V /| |  __/ |
 | .__/|_|  \___/ \_/\_/ |_|\___|_|v2.3.0-22012021
 |_| the handy cloud security tool

 Date: Tue Mar  9 08:08:31 UTC 2021

 Colors code for results: 
 INFO (Information), PASS (Recommended value),  WARNING (Ignored by whitelist),  FAIL (Fix required),  Not Scored 

 10.0 HIPAA Compliance - ONLY AS REFERENCE - [hipaa] ****************    

 1.2  [check12] Ensure multi-factor authentication (MFA) is enabled for all IAM users that have a console password (Scored)   

 1.13  [check113] Ensure MFA is enabled for the root account (Scored)   

 2.3  [check23] Ensure the S3 bucket CloudTrail logs to is not publicly accessible (Scored)   

 2.6  [check26] Ensure S3 bucket access logging is enabled on the CloudTrail S3 bucket (Scored)   

 2.7  [check27] Ensure CloudTrail logs are encrypted at rest using KMS CMKs (Scored)   

 2.9  [check29] Ensure VPC Flow Logging is Enabled in all VPCs (Scored)   

 7.18 [extra718] Check if S3 buckets have server access logging enabled (Not Scored) (Not part of CIS benchmark)    

 7.25 [extra725] Check if S3 buckets have Object-level logging enabled in CloudTrail (Not Scored) (Not part of CIS benchmark)    

 7.2 [extra72] Ensure there are no EBS Snapshots set as Public (Not Scored) (Not part of CIS benchmark)    

 7.5 [extra75] Ensure there are no Security Groups not being used (Not Scored) (Not part of CIS benchmark)    

 7.17 [extra717] Check if Elastic Load Balancers have logging enabled (Not Scored) (Not part of CIS benchmark)    

 7.29 [extra729] Ensure there are no EBS Volumes unencrypted (Not Scored) (Not part of CIS benchmark)    

 7.34 [extra734] Check if S3 buckets have default encryption (SSE) enabled or use a bucket policy to enforce it (Not Scored) (Not part of CIS benchmark)


 3.8  [check38] Ensure a log metric filter and alarm exist for S3 bucket policy changes (Scored)   

 7.3 [extra73] Ensure there are no S3 buckets open to Everyone or Any AWS user (Not Scored) (Not part of CIS benchmark)    

 7.40 [extra740] Check if EBS snapshots are encrypted (Not Scored) (Not part of CIS benchmark)    

 7.35 [extra735] Check if RDS instances storage is encrypted (Not Scored) (Not part of CIS benchmark)    

 1.12  [check112] Ensure no root account access key exists (Scored)   

 1.3  [check13] Ensure credentials unused for 90 days or greater are disabled (Scored)   

 1.5  [check15] Ensure IAM password policy requires at least one uppercase letter (Scored)   

 1.6  [check16] Ensure IAM password policy require at least one lowercase letter (Scored)   

 1.7  [check17] Ensure IAM password policy require at least one symbol (Scored)   

 1.8  [check18] Ensure IAM password policy require at least one number (Scored)   

 1.9  [check19] Ensure IAM password policy requires minimum length of 14 or greater (Scored)   

 2.1  [check21] Ensure CloudTrail is enabled in all regions (Scored)   

 2.4  [check24] Ensure CloudTrail trails are integrated with CloudWatch Logs (Scored)   

 2.8  [check28] Ensure rotation for customer created KMS CMKs is enabled (Scored)   

 3.1  [check31] Ensure a log metric filter and alarm exist for unauthorized API calls (Scored)   

 3.10  [check310] Ensure a log metric filter and alarm exist for security group changes (Scored)   

 3.11  [check311] Ensure a log metric filter and alarm exist for changes to Network Access Control Lists (NACL) (Scored)   

 3.12  [check312] Ensure a log metric filter and alarm exist for changes to network gateways (Scored)   

 3.13  [check313] Ensure a log metric filter and alarm exist for route table changes (Scored)   

 3.14  [check314] Ensure a log metric filter and alarm exist for VPC changes (Scored)   

 3.2  [check32] Ensure a log metric filter and alarm exist for Management Console sign-in without MFA (Scored)   

 3.3  [check33] Ensure a log metric filter and alarm exist for usage of root account (Scored)   

 3.4  [check34] Ensure a log metric filter and alarm exist for IAM policy changes (Scored)   

 3.5  [check35] Ensure a log metric filter and alarm exist for CloudTrail configuration changes (Scored)   

 3.6  [check36] Ensure a log metric filter and alarm exist for AWS Management Console authentication failures (Scored)   

 3.7  [check37] Ensure a log metric filter and alarm exist for disabling or scheduled deletion of customer created KMS CMKs (Scored)   

 3.9  [check39] Ensure a log metric filter and alarm exist for AWS Config configuration changes (Scored)   

 7.92 [extra792] Check if Elastic Load Balancers have insecure SSL ciphers (Not Scored) (Not part of CIS benchmark)   

• 检查某个自定义的profile，region的合规性

$ ./prowler -p custom-profile -r us-east-1

• 只检查某个check的合规，比如我只看check310的，或者多个check

# 一个
$ ./prowler -c check310

# 多个
$ ./prowler -c check310,check722

# 全部检查，除了42,43
$ ./prowler -E check42,check43

# Docker 版
$ docker run -ti --rm --name prowler --env AWS_ACCESS_KEY_ID --env AWS_SECRET_ACCESS_KEY --env AWS_SESSION_TOKEN toniblyx/prowler:latest "-c check310"

• 检查某个合规性group（如hipaa）

$ ./prowler -g hipaa # for hipaa related checks

• 检查某个合规性group的所有条款，除了check 42,43

$ ./prowler -g group4 -E check42,check43

• 输出报告格式为 csv, 或者多种格式都要

$ ./prowler -M csv # 输出为csv
$ ./prowler -M csv,json,json-asff,html # 一次输出为多种格式
$ ./prowler -g gdpr -M csv,json,json-asff  # 讲gdpr的报告输出为多种格式
$ ./prowler -M html # 输出一个排序的动态html报告
$ ./prowler -M mono > prowler-report.txt #输出到文件

• 直接将报告上传到S3桶里（仅支持mono和test格式）

$ ./prowler -M mono | aws s3 cp - s3://bucket-name/prowler-report.txt

• 检查某个合规性group的所有条款，除了check 42,43

$ ./prowler -g group4 -E check42,check43

• 其他更多就自己看help吧：

$ ./prowler -h

USAGE:
      prowler [ -p <profile> -r <region>  -h ]
  Options:
      -p <profile>        specify your AWS profile to use (i.e.: default)
      -r <region>         specify an AWS region to direct API requests to
                            (i.e.: us-east-1), all regions are checked anyway if the check requires it
      -c <check_id>       specify one or multiple check ids separated by commas, to see all available checks use -l option
                            (i.e.: check11 for check 1.1 or extra71,extra72 for extra check 71 and extra check 72)
      -g <group_id>       specify a group of checks by id, to see all available group of checks use -L
                            (i.e.: group3 for entire section 3, cislevel1 for CIS Level 1 Profile Definitions or forensics-ready)
      -f <filterregion>   specify an AWS region to run checks against
                            (i.e.: us-west-1)
      -m <maxitems>       specify the maximum number of items to return for long-running requests (default: 100)
      -M <mode>           output mode: text (default), mono, html, json, json-asff, junit-xml, csv. They can be used combined comma separated.
                            (separator is ,; data is on stdout; progress on stderr).
      -k                  keep the credential report
      -n                  show check numbers to sort easier
                            (i.e.: 1.01 instead of 1.1)
      -l                  list all available checks only (does not perform any check). Add -g <group_id> to only list checks within the specified group
      -L                  list all groups (does not perform any check)
      -e                  exclude group extras
      -E                  execute all tests except a list of specified checks separated by comma (i.e. check21,check31)
      -b                  do not print Prowler banner
      -s                  show scoring report
      -S                  send check output to AWS Security Hub - only valid when the output mode is json-asff (i.e. -M json-asff -S)
      -x                  specify external directory with custom checks (i.e. /my/own/checks, files must start by check)
      -q                  suppress info messages and passing test output
      -A                  account id for the account where to assume a role, requires -R and -T
                            (i.e.: 123456789012)
      -R                  role name to assume in the account, requires -A and -T
                            (i.e.: ProwlerRole)
      -T                  session duration given to that role credentials in seconds, default 1h (3600) recommended 12h, requires -R and -T
                            (i.e.: 43200)
      -I                  External ID to be used when assuming roles (not mandatory), requires -A and -R
      -w                  whitelist file. See whitelist_sample.txt for reference and format
                            (i.e.: whitelist_sample.txt)
      -N <shodan_api_key> Shoadan API key used by check extra7102.
      -V                  show version number & exit
      -h                  this help

更多用法

更多用法，比如跨账号，配合AWS Organization(集团公司，高大上！)，与AWS Security Hub 集成等，请查看官方文档

安民可与行义，危民易与为非