The New Era of DevSecOps

What is DevSecOps?

DevSecOps represents a shift in mindset where security is treated as an integral part of the software development and delivery process. DevSecOps advocates for the integration of security practices, such as code analysis, vulnerability scanning, and continuous monitoring, into every phase of the development process.

In the past, security measures were just an afterthought in the software development lifecycle (SDLC). That afterthought quickly became a wake-up call for developers and security teams when attacks against software supply chains grew by 633% from 2022.

In order to prevent these types of attacks, DevOps had to quickly adopt a shift left approach to better identify and remediate potential security vulnerabilities in real-time.

Shift left in DevSecOps refers to the practice of integrating security earlier in the software development lifecycle. It involves addressing security considerations during the design and development phases and enhances AppSec team security.

A great analogy to describe DevSecOps and the “shift left” approach is to implement the “Five Whys” technique popularized by Toyota as part of their lean manufacturing philosophy to uncover the root cause of a problem.

Think of the SDLC as the manufacturing or production line.

Let's look at an example of a vulnerability discovered in a GitHub repository, where unauthorized users were granted excessive permissions, allowing them to modify and delete critical code.  

Here, we can apply the Five Whys technique to gain a deeper understanding of the root cause of this vulnerability as a result of too many permissions. 

Problem:

Unauthorized users were granted excessive permissions in a GitHub repository, leading to the modification and deletion of critical code. 

❓Why were unauthorized users granted excessive permissions?

‍There was a misconfiguration of permissions in the repository.

❓Why were unauthorized users granted excessive permissions?

There was a misconfiguration of permissions in the repository. 

❓Why was there a misconfiguration of permissions?

The person responsible for setting up the permissions did not have a clear understanding of the appropriate access levels and the principle of least privilege. 

❓Why did the person lack a clear understanding of access levels and the principle of least privilege?

The organization did not provide sufficient training or guidelines on configuring and assigning permissions in GitHub repositories.

❓Why did the organization not provide sufficient training or guidelines?

The organization did not prioritize security training and did not have established procedures or documentation for configuring permissions in GitHub repositories.

❓Why did the organization not prioritize security training and established procedures?

There was a lack of awareness of the potential risks and the importance of proper permissions management within the organization.

Root Cause:

Insufficient awareness, training, and established procedures for configuring permissions in GitHub repositories resulted in misconfigurations and unauthorized users being granted excessive permissions, leading to that vulnerability. 

Solution:

Implement best practices for securing your repository and ensure that the principle of least privilege is enforced to prevent unauthorized access.

DevSecOps Best Practices

Adopting a Shift Left Mindset

‍Shifting left requires a change in mindset and culture within an organization. It involves breaking down traditional silos between teams. The rewards of shifting left are catching vulnerabilities early on before they progress further down the SDLC and make it into production. One of the most critical vulnerabilities is found in hardcoded secrets. Git Guardian recently uncovered 10 million secrets hidden in 1 billion commits in public repositories. GitHub as described in the State of Secrets Sprawl report. And any of those vulnerabilities could have easily gone unnoticed until a major breach occurred. 

Shared Responsibility Belongs to Everyone

Shared responsibility promotes transparency and accountability within the organization. With shared responsibility, each team member actively participates in identifying and addressing security issues. This creates less friction and promotes a company culture of continuous testing and improvement. Here, everyone works together like a well-oiled machine, greatly improving team communication and accelerating delivery times.

Implement Security as Code: Treat security infrastructure and controls as code artifacts. Use version control, automated testing, and deployment techniques to ensure consistent and repeatable security configurations. Review and enforce security policies across your organization.

Integrate Automation Tools

Leverage automation tools such as SAST, DAST, Secrets Detection, and Software composition analysis (SCA)to streamline security processes. Automated security processes can be easily integrated into CI/CD pipelines and scaled to accommodate larger codebases or evolving infrastructure. Security as Code also eliminates manual configuration inconsistencies and reduces human error.

Continuous Monitoring

‍Continuous monitoring should be performed at all times. By analyzing security events and metrics, organizations can gain insights into the effectiveness of security controls, identify patterns or trends, and make data-driven decisions to enhance their security posture. When any suspicious behavior and anomalies are detected, automated alerting systems immediately notify security teams, enabling rapid incident response to minimize exposure.    

Heading in the Right Direction: Importance of Shift Left Security

Organizations Are constantly under tremendous pressure to meet tight deadlines and ship products quickly to market. By prioritizing speed over security, you could easily overlook potential vulnerabilities and risks during the development process.   

Fixing Vulnerabilities at a later stage requires extra work, extensive testing, and potential disruptions to the production environment, resulting in higher costs and delayed time to market. The shift right approach simply did not work.    

Shifting security left enables the early identification, prioritization, and mitigation of vulnerabilities. This reduces the risk of a critical vulnerability slipping through the cracks of the SDLC and reaching production environments.  

The New Way of Securing the SDLC

The Secure Software Development Life Cycle (SSDLC) is a systematic approach to integrating security into the software development process from code to production. Each phase of the SDLC plays a crucial role in ensuring the development of secure and resilient software. 

Here is a step-by-step breakdown of the main phases of the SDLC using a blueprint to construct a house.  

Planning Phase

‍This phase is similar to the architectural planning and design phase of building a house. It involves gathering requirements, understanding the needs of future occupants, and determining the overall vision and scope of the house. Just like in software development, this phase lays the foundation for the entire project, identifying the key features and functionality required. 

Analysis Phase‍

In the context of building a house, the analysis phase corresponds to evaluating the feasibility of the project. It involves assessing factors such as the available resources, budget, land restrictions, and local building codes. This phase helps identify any potential challenges or constraints that may impact the construction process. This is similar to gathering information from homeowners about their desired house layout, number of rooms, architectural style, and other specific requirements. Think of it as building codes and zoning regulations that need to be taken into account. 

Design Phase

‍Similar to an architect creating detailed blueprints for a house, the design phase in the SDLC involves creating the system architecture and designing the components and interfaces. It outlines the structure and layout of the software solution, just as an architectural plan defines the layout, floor plans, and aesthetics of a house. Software architects define the system architecture, which outlines the overall structure and components of the software. This involves determining the major modules, subsystems, and their interactions. It would be like an architect designing the structural framework of the house, including the foundation, walls, roof, and supporting systems.  

Development Phase

‍The development phase can be compared to the actual construction of the house. Developers conduct unit testing to ensure component functionality and quality. This involves writing and executing test cases. When building a house, construction quality checks ensure proper installation and functioning of elements like plumbing and electrical connections. Developers write the code, similar to how workers construct the physical structure of the house. The coding process follows the design specifications, building the software component by component, just as construction workers build the house using bricks, wood, and other materials. 

Testing Phase

‍In the SDLC, the testing phase ensures that the software functions as intended and meets the specified requirements. This phase is analogous to quality inspections and tests carried out during the construction of a house. Just as builders inspect the structural integrity, plumbing, and electrical systems of a house, software testers validate the functionality, performance, security, and usability of the software. Usability and regression testing are performed during this phase.  

Usability Testing would be like inspecting the layout and design of a house to ensure it provides comfort for those living inside. Regression testing ensures that changes or modifications made to the software did not unintentionally introduce new defects or negatively impact existing functionality. 

Deployment/Maintenance Phase

‍Once the house is built, it needs to be prepared for occupancy. Similarly, in the SDLC, the deployment phase involves releasing the software solution and making it available for users. Ongoing maintenance and support are also required, similar to the maintenance required for a house, such as repairs, upgrades, and regular inspections to ensure everything continues to function smoothly.  

DevSecOps Tools: The Essentials

DevSecOps teams must have a swiss-army knife of tools to automate security testing and validation processes, scan open-source libraries and repositories for compromised dependencies, and yet, somehow, someway, manage to maintain compliance with constantly changing industry regulations. And if that’s not hard enough, let’s throw third-party vendors into the mix.   

For Instance, let's say a DevOps team relies on a third-party library for handling user authentication in their application. If a vulnerability is discovered in that library and it is exploited by attackers, it could lead to unauthorized user accounts or the exposure of sensitive user data. 

In This scenario, these DevSecOps tools can play a crucial role in preventing such breaches.  

Here's how. 

Software Composition Analysis (SCA) Tools

Research taken from the Open-Source Security and Risk Analysis (OSSRA) report showed that 84% of codebases contained an open-source vulnerability. SCA tools can scan the organization's codebase and identify any known vulnerabilities present in third-party libraries or components.  

SEA tools not only identify security vulnerabilities but also help organizations manage license compliance. By integrating SCA tools into the CI/CD pipeline, the development team can automatically detect vulnerabilities in third-party libraries. 

IaC Scanning

IaC scanning tools automate the process of analyzing and evaluating infrastructure code against security best practices and predefined security policies. This eliminates the need for manual code reviews and helps prevent misconfigurations. During the scanning process, the infrastructure code is thoroughly analyzed to identify common security flaws, such as weak access controls, insecure network configurations, hardcoded credentials, outdated software versions, or misconfigured security groups.

 OWASP recently updated the rankings of their Mobile Top 10 with Security Misconfiguration and Improper Credential Usage topping the list. MITRE has the Use of Hard-coded Credentials (CWE-798) as part of their Top 25 Most Dangerous Software Weaknesses.

Find And fix hard coded secrets fast and avoid any IaC misconfigurations by enhancing your code security posture.  

Patch and Dependency Management: DevSecOps tools can help manage dependencies and ensure that the latest, patched versions of third-party libraries are used. By automating the process of updating and managing dependencies, organizations can quickly adopt new versions that address known vulnerabilities, reducing the risk of a breach resulting from outdated or vulnerable components. 

Threat Intelligence

DevSecOps tools that offer continuous monitoring and threat intelligence capabilities can help organizations stay informed about the latest vulnerabilities affecting third-party components. Threat intelligence provides contextual information about the threat landscape, including indicators of compromise (IOCs), malicious IP addresses, domains, and malware signatures. This information helps DevSecOps teams prioritize and focus their efforts on addressing the most relevant and impactful security risks.  

Honeytokens: Accelerating CI/CD Security

CI/CD security ensures the seamless and efficient delivery of incremental code changes from a developer's workstation to production environments. The CI/CD pipeline integrates security throughout the software development lifecycle.  

CI/CD pipeline automates the process of code compilation, testing, and deployment. If The code being processed contains vulnerabilities or is insecure, those weaknesses can be propagated quickly throughout the pipeline, leading to potential security breaches. A leading vulnerability found in the CI/CD pipeline is insecure secrets management.

 Git Guardian uncovered over 10 million leaked secrets that were detected in public repositories. Exposed secrets allow attackers to escalate their privileges within the CI/CD pipeline environment. They can use stolen credentials or access tokens to move laterally across the pipeline, manipulating sensitive data, injecting malicious code, or compromising other components. This can lead to the spread of malware, unauthorized code deployments, or the introduction of vulnerabilities into the SDLC.

 CI/CD pipelines typically involve different environments, such as development, testing, staging, and production. Each environment should have its own set of secrets and configuration values. Instead of storing secrets directly in code repositories or configuration files, secrets should be protected and kept in secure vaults or key management systems. The CI/CD pipeline can then retrieve the secrets at runtime.

What Are Honeytokens?

Honeytokens Are decoy assets that are intentionally created to detect and track unauthorized access. Honeytokens can take various forms, such as fake user accounts, false credentials, or bogus files. They are typically scattered throughout an organization's systems, networks, or applications, appearing as legitimate targets to trap potential attackers.

Honeytokens can be embedded within the code or artifacts that are part of the CI/CD pipeline. For example, dummy credentials or API keys can be included in configuration files or scripts. These honeytokens are not used in production but are designed to be attractive to attackers. When honeytokens are accessed, they can provide valuable insights into the tactics, techniques, and procedures (TTPs) employed by attackers and the specific points of vulnerability within the CI/CD pipeline.  

Third-party CI/CD security tools such as CircleCI, Jenkins, and Travis CI often require granting access permissions to external entities. Honeytokens allow you to validate the effectiveness of access controls and permissions granted to the third-party integrations. If a honeytoken is accessed by an unauthorized user, it indicates a failure in your access controls and prompts you to strengthen your permissions model.  

Circling Back (had to throw that in) to our example of the Five Whys Git permissions example in the context of shifting left in a DevSecOps model. Incorporating honeytokens into your CI/CD pipeline strategy can send potential intruders away and give yourSELF the added layer of security and your dev team a good night’s sleep.

8 Questions to Ask When Choosing SCA (Software Composition Analysis) Tools

SCA tools help identify and manage open-source and third-party components, providing visibility into their security vulnerabilities, and license compliance. 

SCA tools focus on open-source software security by analyzing the security posture of the components. If left unsecured, open-source packages can potentially expose an organization to critical vulnerabilities. SCA security is essential for preventing attacks.      

When choosing an SCA tool it’s important to take the following into consideration:  

• Does this SCA tool integrated with your existing development and DevOps processes?
• Does the tool provide comprehensive license management capabilities?
• How does the tool handle false positives and false negatives in vulnerability detection?
• Does the tool offer remediation guidance?
• What level of support and documentation does the tool provide?
• Can it seamlessly integrate with your existing development and CI/CD toolchain?
• Can the tool analyze both direct and transitive dependencies?
• What level of customization and configuration does the SCA tool offer? 

SAST and Source Code Analysis: Uncovering the Hidden Threat Surface

SAST and source code analysis tools integrate seamlessly into the development workflow, providing developers with real-time feedback and actionable recommendations.  

Both tools can be integrated into CI/CD pipelines, IDEs, and version control systems, enabling developers to address security vulnerabilities and code quality issues in real-time, rather than relying solely on manual reviews. 

SAST scanning and Source Code Analysis Tools aim to minimize false positives, ensuring that the reported vulnerabilities are genuine threats. Both tools spare IT teams from alert fatigue and minimize the noise, enabling them to prioritize and remediate critical threats based on the findings much earlier in the SDLC.

SAST vs SCA

Although FAST and SCA serve different purposes, they complement each other in identifying and mitigating security risks. Here are a few differences and advantages of SAST and SCA. 

Using Both SAST and SCA tools together greatly enhances an organization’s AppSec posture. 

SAST vs DAST

SAST and DAST address different stages of the software development lifecycle. SAST is well-suited for early stages, such as during development and unit testing, where it can provide developers with immediate feedback on potential vulnerabilities and enable them to fix issues before they become more costly to address.  

DAST is typically used in later stages, such as during integration testing, user acceptance testing, and production monitoring. Although both provide a Ying-Yang security synergy to the SDLC. Now, for the bad news, SAST and DAST simply aren’t enough to prevent hard coded secrets from public repositories.  

“The fact is, for static analysis tools, this huge threat surface is invisible because only the current state of your codebase is scanned.”  

The Importance of an SBOM in DevSecOps

An SBOM or Software Bill of Materials serves as a foundational document that helps organizations gain visibility into the software supply chain, assess security risks, and promotes effective vulnerability management in DevSecOps practices.

Without SBOM, it would be challenging to ensure the security and reliability of the software supply chain. Imagine driving a car without knowing the condition of its engine, brakes, or other critical components. Not suggested. An SBOM goes "under the hood", enabling security professionals with the "tools" they need to fix and remediate vulnerabilities at the far-left side of the SDLC. 

Containers and Kubernetes Security: Leave Nothing Unscanned

Container Security: 4 Best Practices

Container Security is a critical part of the software supply chain and in DevSecOps as it ensures the protection of applications.  

Here Are four best practices for securing Docker containers: 

1. Harden Container Hosts: Secure the underlying host system that runs Docker containers. Keep the host's operating system and software up to date with the latest security patches.
2. Enforce Secure Configuration: This includes disabling and removing unnecessary services, enabling security features like Seccomp, and using strong authentication mechanisms for container access. Check out our step-by-step guide to creating a Seccomp Filter Generation Workflow here.
3. Secure Image Management: Emphasizes the use of trusted and verified container images from reliable sources. Regularly update and patch images to address known vulnerabilities. Container image scanning tools such as Docker helps identify security issues such as malware within images before deployment.
4. Secure Container Registry: Ensure proper authentication and access controls are in place to prevent unauthorized access to container images. Utilize encryption for secure transmission of images between the registry and Docker hosts and verify the integrity of stored images.    

3 Proven Strategies to Accelerate Kubernetes Security

Kubernetes Provides a powerful container orchestration platform for managing and deploying containerized applications. Here are a few ways to secure your K8s:

1. Cluster Hardening

Cluster Hardening: Regularly apply security patches and updates to the cluster components, including the control plane and worker nodes. Enforce network policies to control inbound and outbound traffic between pods, namespaces, and external resources. Implement firewalls, network segmentation, and proper ingress/egress controls to limit exposure.

‍2. Image Security

Container images used in K8s Should be scanned for vulnerabilities and verified for integrity. Employ a container image registry that enforces access controls and implements image signing and verification mechanisms.

‍3. Secure Secrets Management‍

Make certain you are managing sensitive information such as passwords, API keys, tokens, and certificates using Kubernetes Secrets. Avoid hard-coding sensitive data into application code or configuration files.

Security Monitoring and Configuration: The 'Right' Part of the Wheel

What Could something as “harmless” as a misconfigured AWS S3 Bucket Cause? Only 3TB of extremely sensitive airport data (1.5+ million files) leaked.  

Securing AWS S3 buckets becomes a much simpler process with Infrastructure as Code Security in place. Security monitoring goes beyond the “right” side of the DevSecOps loop. Customers value transparency. Security Monitoring demonstrates a commitment to your customers, from the far left side of the wheel all the way to production. Code reviews contribute to improved software security and ensure that the software being delivered meets the quality standards of the customer. When added into a DevSecOps model, both dev teams and customers benefit tremendously. 

FAQ

What are the 4 components of DevSecOps?

‍‍DevSecOps is comprised of development (Dev), operations (Ops), and security (Sec). It emphasizes a shared responsibility and prioritized the integration of security into the entire software development lifecycle.

What is an example of DevSecOps?

Integrating automated security testing into the CI/CD pipeline to continuously scan and monitor code for vulnerabilities and by conducting regular code reviews.

What are the DevSecOps tools?

‍ DevSecOps tools include static application security testing (SAST), dynamic application security testing (DAST), software composition analysis (SCA), IaC scanning, secrets detection, container security, CI/CD pipeline security, and honeytokens.