00:00:01:10 - 00:00:02:21 Good morning. Good afternoon. 00:00:02:21 - 00:00:03:21 Good evening. 00:00:03:21 - 00:00:07:15 This is the team that Michaela introduced with visuals. 00:00:07:15 - 00:00:09:19 I’m a bit, under the weather today. 00:00:09:19 - 00:00:14:09 So I think I'd better keep just my voice into the meeting today. 00:00:14:09 - 00:00:16:05 We are going to discuss today. 00:00:16:05 - 00:00:19:23 Another point on our journey with leveraging, OSCAL. 00:00:20:01 - 00:00:22:16 We started the journey. I'd say five years ago. 00:00:22:16 - 00:00:25:23 We have presented a couple of times on, on this forum. 00:00:26:03 - 00:00:28:05 I know that typically we talk about compliance. 00:00:28:05 - 00:00:31:23 So I wanted to just to take a few minutes share with you that we use, 00:00:32:07 - 00:00:35:00 OSCAL and technology that we present here. 00:00:35:00 - 00:00:37:22 In, many aspects of, 00:00:37:22 - 00:00:41:03 the guardrails for us and for our clients. 00:00:41:03 - 00:00:46:21 That include, of course, the, security, requirements, compliance requirements 00:00:46:21 - 00:00:51:07 when those, become policy and they're mandatory and in general, 00:00:51:07 - 00:00:55:19 in any, best practices, they all follow the same, structure. 00:00:55:19 - 00:00:57:08 The difference is that when we are talking 00:00:57:08 - 00:01:01:02 strictly about compliance, those guardrails become policy. 00:01:01:03 - 00:01:03:01 They must be enforced. 00:01:03:01 - 00:01:07:20 Besides those various aspects of leveraging, guardrails, 00:01:07:20 - 00:01:11:01 we also apply this at any layer in the stack 00:01:11:01 - 00:01:14:08 from the infrastructure data, AI application and processes. 00:01:14:12 - 00:01:19:00 So this is where our focus is recently in compliance for AI. 00:01:19:02 - 00:01:21:11 Again based on the OSCAL structure. 00:01:21:11 - 00:01:24:14 And as far as we applied it, we haven't 00:01:24:14 - 00:01:27:14 perceived need for any change to the, current, schema. 00:01:27:16 - 00:01:29:13 It can be applied in different domains. 00:01:29:13 - 00:01:33:02 So we applied it in security and business and finance and so on. 00:01:33:06 - 00:01:36:11 And the life cycle that we, leverage across 00:01:36:11 - 00:01:40:19 all those domains and stack and so on includes those four major steps. 00:01:40:22 - 00:01:44:16 The definition and authoring of the controls, the implementation 00:01:44:16 - 00:01:47:22 of the controls, assessment and then audit and reporting. 00:01:48:00 - 00:01:51:17 And, the talk today is mainly related to step number three. 00:01:51:20 - 00:01:56:09 We have presented in the past our technology to use compliance as code. 00:01:56:09 - 00:01:57:19 OSCAL to 00:01:57:19 - 00:02:00:05 Support the definition and offering. 00:02:00:05 - 00:02:05:13 And we presented also a tool to bridge between step one and step three. 00:02:05:15 - 00:02:08:14 And the team is going to talk about compliance to policy. 00:02:08:14 - 00:02:12:23 And in this context we are going to see AI for compliance. 00:02:13:03 - 00:02:16:00 So in the context of OSCAL. 00:02:16:00 - 00:02:19:06 We have those two sides on the compliance administration 00:02:19:06 - 00:02:21:03 and the policy administration. 00:02:21:03 - 00:02:22:17 Right, left and right. 00:02:22:17 - 00:02:28:16 They use complementary technology and languages and processes. 00:02:28:19 - 00:02:30:15 On the compliance administration side, 00:02:30:15 - 00:02:34:10 we have all the compliance as code and OSCAL and the authoring. 00:02:34:10 - 00:02:37:23 And in this context we have presented the trestle and the general authoring. 00:02:37:24 - 00:02:41:08 then on the right side we have the policy validation tools. 00:02:41:10 - 00:02:45:17 We've seen, in Kubernetes, we have Kyverno and or gatekeeper. 00:02:45:17 - 00:02:48:08 We have an open source, audit tree. 00:02:48:08 - 00:02:52:11 Ansible and of course, the, commercial, CSP and so on. 00:02:52:11 - 00:02:57:07 And the bridging between those two is, supported in OSCAL 00:02:57:11 - 00:02:59:11 through, component definition. Right. 00:02:59:11 - 00:03:03:05 So we have an opinionated implementation in trestle to bridge 00:03:03:05 - 00:03:07:05 between the, compliance code and the checks policy as code. 00:03:07:05 - 00:03:10:07 And we build on that to support the automation. 00:03:10:09 - 00:03:13:22 So this is this is from the last meeting just to, remind ourselves 00:03:13:22 - 00:03:16:20 the tools, OSCAL is the language trestle is the SDK. 00:03:16:20 - 00:03:20:01 Agile authoring is the automated workflow for the authoring 00:03:20:01 - 00:03:24:22 and then compliance to policy is where, we are able to administer, 00:03:24:24 - 00:03:30:22 authored artifacts, baselines, component definition to runtime to its environment. 00:03:31:00 - 00:03:34:04 And when I say runtime, it doesn't necessarily mean, deployed. 00:03:34:04 - 00:03:39:08 It can be also pre-deployment, but it runs against actual environment 00:03:39:08 - 00:03:44:10 and artifacts versus the left side, which is more consisting of libraries 00:03:44:10 - 00:03:45:10 that can be leveraged 00:03:45:10 - 00:03:49:12 for the instantiations on the, the implementations of the environments. 00:03:49:12 - 00:03:52:06 So with that, I'm going to stop sharing. 00:03:52:06 - 00:03:55:13 And, Yuji I think you can take it, and go into the details 00:03:55:13 - 00:03:58:15 of the compliance to policy and our journey. 00:03:58:15 - 00:04:00:13 Okay. Yeah. Sure. 00:04:00:13 - 00:04:03:01 Let me share my screen. So. Hi. 00:04:03:01 - 00:04:04:08 My name is Takumi Yanagawa 00:04:04:08 - 00:04:08:19 I'm a, a founder of Compliance-to-Policy Project in OSCAL compass. 00:04:08:21 - 00:04:13:16 Project as Anca, introduced the compliance-to-policy, we call C2P 00:04:13:23 - 00:04:18:20 is the essential component to bridge both the levels of compliance framework 00:04:18:20 - 00:04:22:20 and the individual, policy, validation and, engines. 00:04:22:23 - 00:04:25:08 So to bridge, both levels 00:04:25:08 - 00:04:28:20 a plugin design is, essential, So I would like to, 00:04:28:23 - 00:04:32:19 briefly explain how C2P is, designed and worked. 00:04:33:03 - 00:04:35:19 here is the, high level conceptual diagrams. 00:04:35:19 - 00:04:38:19 The input of C2P is OSCAL. 00:04:38:19 - 00:04:42:17 Primary input is OSCAL component definitions from starting OSCAL component 00:04:42:17 - 00:04:43:03 definition 00:04:43:03 - 00:04:47:07 And finally it produces OSCAL assessment or results. 00:04:47:10 - 00:04:52:19 So the input interface is OSCAL's on the other hand, from the component definitions. 00:04:52:19 - 00:04:56:09 C2P produces, technology specific policies. 00:04:56:13 - 00:04:59:13 For example, if you use Ansible, then 00:05:00:04 - 00:05:04:04 generates Ansible playbooks, and then it's deployed to, Ansible 00:05:04:04 - 00:05:08:07 to validate the configurations of VM networks. 00:05:08:07 - 00:05:11:07 If we use, Kyverno, is against the Kubernetes. 00:05:11:07 - 00:05:14:14 Then Kyverno policy is generated from component definitions. 00:05:14:16 - 00:05:18:04 So this is the, direction from compliance to policies. 00:05:18:06 - 00:05:21:05 similarly, the, results from policy 00:05:21:05 - 00:05:24:05 validations from individual engines. 00:05:24:09 - 00:05:27:16 Also converted to OSCAL assessment results. 00:05:27:21 - 00:05:32:14 This is the higher level of workflows of C2P 00:05:32:17 - 00:05:37:04 but how I would like to, explain the next level with details. 00:05:37:12 - 00:05:40:01 about how to generate policies 00:05:40:01 - 00:05:42:24 Because this is important for our journey 00:05:42:24 - 00:05:46:18 to use, Gen-AI and LLM for further automation. 00:05:46:22 - 00:05:51:06 So the current, origin of C2P consists of two layers. 00:05:51:17 - 00:05:54:08 One is called C2P core. 00:05:54:08 - 00:05:59:18 This component is responsible of interpreting OSCAL's finally extract 00:05:59:18 - 00:06:05:16 the mapping between complex controls and technology specific rules and policy. 00:06:05:20 - 00:06:09:06 with these mappings the next layer is plugins. 00:06:09:09 - 00:06:12:12 For example, if you use Ansible then Ansible plugin 00:06:12:13 - 00:06:15:23 is picking the mappings and then retrieve 00:06:16:12 - 00:06:19:20 corresponding policies because many security engines 00:06:19:20 - 00:06:22:20 such as Ansible open scalp, Kubernetes 00:06:22:24 - 00:06:26:16 has maintain policy collections in their 00:06:26:16 - 00:06:28:18 Repository or registries. 00:06:28:18 - 00:06:31:16 For example, in the case of Ansible You can go to Ansible 00:06:31:16 - 00:06:34:21 Galaxy and filter by level security. 00:06:34:21 - 00:06:37:24 Then you can find there are many policy Ansible playbooks. 00:06:38:04 - 00:06:42:12 So the plugin responsible of retrieving matching policies 00:06:42:14 - 00:06:45:13 with the, controllers from the controller mappings 00:06:45:13 - 00:06:48:18 and then deliver the policy to the target resources. 00:06:48:19 - 00:06:52:17 Also, the, component definition contains the parameters. 00:06:52:17 - 00:06:56:13 So these parameters are also expanded by core and pass to 00:06:56:13 - 00:06:59:24 the plugin and also C2P core select plugins. 00:07:00:05 - 00:07:03:10 for example component and corresponding validation 00:07:03:10 - 00:07:07:06 is Ansible, then C2P core select Ansible plugin. 00:07:07:06 - 00:07:12:13 If the Kubernetes is the component then C2P plugin select kyverno plugin. 00:07:12:13 - 00:07:16:04 these two rails the responsibilities and similarly 00:07:16:13 - 00:07:22:10 reverse directions from the policy result to normalized OSCAL assessment result. 00:07:22:10 - 00:07:26:23 This direction is, more straightforward because each engine 00:07:26:24 - 00:07:30:17 has their own format of the, policy validation results. 00:07:30:20 - 00:07:36:03 So it's straightforwardly, programing, converting to normalized format 00:07:36:08 - 00:07:39:21 and then C2P core aggregates the normalized results 00:07:39:24 - 00:07:43:11 to OSCAL assessment result using component definition 00:07:44:07 - 00:07:49:22 This is the, details about how to generate policy that current C2P. 00:07:50:01 - 00:07:54:09 However, with this, C2P, we can seamlessly integrate 00:07:54:15 - 00:07:58:10 OSCAL framework with the, actual implementation of checks, 00:07:58:13 - 00:08:03:17 policy check, control checks, using security tools or security programs. 00:08:04:04 - 00:08:06:09 sorry for interrupting. Just just a second. 00:08:06:09 - 00:08:10:16 policy is a very loaded word, and, we distinguish between, 00:08:10:21 - 00:08:15:05 capital P, Policy, so to speak, or the organizational policy is not that 00:08:15:05 - 00:08:19:12 those large, bodies of, high level, Corporate Policies. 00:08:19:12 - 00:08:19:20 Right. 00:08:19:20 - 00:08:22:24 The the the starting point of the directives versus the policy, 00:08:22:24 - 00:08:26:06 automation and policy as code, like small “p” policy 00:08:26:06 - 00:08:30:02 in all our presentation here, we are referring to the second right. 00:08:30:02 - 00:08:33:09 We are not talking about the large bodies of corporate policies, 00:08:33:09 - 00:08:36:22 just the small p policies and policy automation and policy as code. 00:08:36:23 - 00:08:37:22 Thank you. 00:08:37:22 - 00:08:39:15 Please go ahead, Yana 00:08:39:15 - 00:08:41:00 Thank you for explaining, Anca 00:08:41:00 - 00:08:46:04 So, but, if we encounter a new control, compliance requirement or controls, 00:08:46:06 - 00:08:51:24 we have to create, corresponding policy and store in policy repositories. 00:08:52:02 - 00:08:56:00 This is very human intensive tasks because, implementing 00:08:56:09 - 00:08:59:12 of policy needs to have domain specific 00:08:59:12 - 00:09:02:14 knowledge and also policy language. 00:09:02:14 - 00:09:07:05 For example, in the case of Ansible, it's Ansible language in the case of OPA. 00:09:07:07 - 00:09:11:09 they know they are already aware of our policy language. 00:09:11:16 - 00:09:15:00 wanted to remove this kind of human intensive task. 00:09:15:00 - 00:09:19:17 So that's why, Gen-AI had the capability to generate 00:09:19:22 - 00:09:22:22 such a programing code, as a policy 00:09:22:22 - 00:09:28:02 and also can have the domain knowledge by training or prompt tuning. 00:09:28:04 - 00:09:33:15 So leveraging Gen-AI, we can, achieve to fully automate policy generation. 00:09:33:22 - 00:09:37:10 So this picture is, illustrating the, 00:09:38:10 - 00:09:40:02 this C2Ps. 00:09:40:02 - 00:09:45:04 why we use Gen-AI, another reason is, we can leverage approach. 00:09:45:04 - 00:09:49:19 the existing C2P retrieves a policy from policy repositories. 00:09:49:20 - 00:09:53:06 So if we new controls, we cannot find, matching 00:09:53:09 - 00:09:56:08 policy, but, similar policy can be retrieved. 00:09:56:08 - 00:10:00:01 So using these similar policies as a sample policies 00:10:00:04 - 00:10:05:22 and control descriptions, those put together into a prompt as the context 00:10:06:01 - 00:10:11:01 and then request LLM, to generate, for example, Ansible policy from this context. 00:10:11:04 - 00:10:14:07 LLM generates, Ansible playbook as a policy. 00:10:14:07 - 00:10:18:13 This is our Gen-AI based C2P. 00:10:18:19 - 00:10:22:00 So I’d like to show an example prompt. 00:10:23:18 - 00:10:26:16 So this is the example prompt 00:10:26:16 - 00:10:29:14 the so the final goal is like this. 00:10:29:14 - 00:10:32:06 So please provide Ansible playbooks. 00:10:32:06 - 00:10:35:03 And borrowing the sentence is the compliance 00:10:35:03 - 00:10:36:01 technology specific rules for example this is against the RHEL machine 00:10:36:01 - 00:10:37:00 technology specific rules for example this is against the RHEL machine 00:10:37:00 - 00:10:38:22 technology specific rules for example this is against the RHEL machine 00:10:39:02 - 00:10:42:13 and security check on the SSH and reads 00:10:42:16 - 00:10:46:10 sample playbook information, relevant information. 00:10:46:12 - 00:10:49:09 These are coming from, policy repositories. 00:10:49:09 - 00:10:52:18 And then LLM generates following the output. 00:10:53:03 - 00:10:56:22 The output is the corresponding, Ansible playbooks. 00:10:57:02 - 00:11:00:01 So this is, the process of this, part, 00:11:00:01 - 00:11:04:09 I would like to, show another, example through demonstrations. 00:11:04:09 - 00:11:06:10 This is end to end the demonstrations. 00:11:06:10 - 00:11:10:09 starting from component definition, this is will present it as Excel, 00:11:10:11 - 00:11:13:21 but it's maintained as component definition Json. 00:11:13:23 - 00:11:17:04 And starting from component definitions it contains mapping 00:11:17:04 - 00:11:20:04 between compliance controls and our policies. 00:11:20:04 - 00:11:23:13 And it's converted C2P prosperous generate 00:11:23:13 - 00:11:26:13 the policies corresponding to each rules. 00:11:26:14 - 00:11:31:15 And in this demo we, check my RHEL machines with Ansible. 00:11:31:15 - 00:11:35:07 So play generated playbooks run in the Ansible. 00:11:35:15 - 00:11:36:16 run that and the 00:11:36:16 - 00:11:40:07 that result would be, converted to OSCAL assessment to result. 00:11:40:12 - 00:11:44:24 So first we prepare component definitions with. 00:11:45:06 - 00:11:49:03 And then feed it in C2P prosperous. 00:11:49:05 - 00:11:50:05 Now it's calling the LLMs 00:11:51:16 - 00:11:54:17 and it's generated the Ansible playbooks. 00:11:54:21 - 00:11:58:04 The generated Ansible playbooks looks like this one. 00:11:58:09 - 00:12:02:24 And we have two Ansible playbooks it will be deployed to Ansible runners. 00:12:03:03 - 00:12:05:14 And the check finished. 00:12:05:14 - 00:12:10:08 And this is the, standard output of Ansible results. 00:12:10:15 - 00:12:12:14 So it contains, reasons. 00:12:12:14 - 00:12:15:18 And results of success fails or error. 00:12:15:21 - 00:12:20:05 So passing this result by C2P existing C2P 00:12:20:07 - 00:12:23:07 then it generates OSCAL assessment result. 00:12:24:04 - 00:12:28:15 The OSCAL Assessment Results, in this demo I beautify the OSCAL assessment results 00:12:28:15 - 00:12:31:03 as markdown format. 00:12:31:03 - 00:12:34:24 But the actual assessment result looks like this one. 00:12:37:05 - 00:12:39:01 So while you are looking for this Yana, 00:12:39:01 - 00:12:42:07 I'll take a second to make a comment on the, 00:12:42:07 - 00:12:45:16 opinionated Trestle implementation for the component definition. 00:12:45:16 - 00:12:48:19 We are all OSCAL valid, artifacts. Right? 00:12:48:19 - 00:12:53:05 So, passing against this, validating against the schema and so on, what we mean 00:12:53:05 - 00:12:58:07 by opinionated and to, help bridge between the compliance and policy, 00:12:58:07 - 00:13:02:20 we use the component definition control, implement, for the controls 00:13:02:20 - 00:13:06:21 declaration, the properties at that level in the schema 00:13:06:21 - 00:13:11:22 in the component definition file, to declare the rules to associate 00:13:11:22 - 00:13:15:02 the technical rules of each component for the control. 00:13:15:02 - 00:13:19:01 We also declare the assessment tools 00:13:19:03 - 00:13:24:10 as components and we map again at each control level, the rules. 00:13:24:10 - 00:13:25:15 are typically in English. 00:13:25:15 - 00:13:29:04 take the example of CS benchmark for instance, we associate them 00:13:29:04 - 00:13:33:15 with the ID of the policy engine check script. 00:13:33:17 - 00:13:38:13 So this mapping from in the component definition from the controls, 00:13:38:14 - 00:13:41:08 that are associated with the software to the component 00:13:41:08 - 00:13:44:09 technical rules that reflect that control. 00:13:44:09 - 00:13:48:22 And, farther, to the IDs of the policy engine check 00:13:48:22 - 00:13:51:22 whether the policy engine is declarative or imperative. 00:13:52:00 - 00:13:52:19 It doesn't matter. 00:13:52:19 - 00:13:55:00 Or even if there are manual assessments. Right. 00:13:55:00 - 00:13:58:15 So this is not only for mapped to those assessment IDs. 00:13:58:15 - 00:14:02:19 And now when, what the Yana is showing when we are getting back those, 00:14:02:19 - 00:14:05:19 checks, they have a well identified ID with the posture. 00:14:05:22 - 00:14:07:14 Now we are able to traverse. 00:14:07:14 - 00:14:09:02 And Yana is showing an example here for RHEL. 00:14:09:02 - 00:14:10:15 And Yana is showing an example here for RHEL. 00:14:10:15 - 00:14:16:19 We have the, the rules with, the, SSH marks start up, you recognize those rules. 00:14:16:19 - 00:14:20:20 If you are, right, with the, VM expert and now the C2P is able 00:14:20:20 - 00:14:24:06 to traverse back, through the component definition from the checks 00:14:24:06 - 00:14:28:02 posture and aggregate up to the rules and the controls. 00:14:28:02 - 00:14:30:04 So this is the core of the automation. 00:14:30:04 - 00:14:34:08 that allows, For the, relationship between compliance and policy. 00:14:34:08 - 00:14:36:14 Again, this is small policy, right? 00:14:36:14 - 00:14:38:20 policy is code back and forth. Please go ahead. 00:14:38:20 - 00:14:39:04 Yeah. 00:14:39:04 - 00:14:42:15 yeah, this is component definition and the mapping of the rules and controls 00:14:42:20 - 00:14:47:03 and from aggregating these rules and the policy results. 00:14:47:07 - 00:14:50:21 it's stored in the observations field in the assessment 00:14:50:21 - 00:14:53:14 result with row evidence 00:14:53:14 - 00:14:54:17 And the results. 00:14:54:17 - 00:14:58:19 So this is the end of C2P workflows. 00:14:58:19 - 00:15:03:05 So now we trying more automating approach 00:15:03:05 - 00:15:07:11 using Agentic AI so we are transitioning from 00:15:07:11 - 00:15:11:23 C2P rule based and GenAI based and now agentic approach. 00:15:11:23 - 00:15:16:11 So this is explain to us but the agent approach, input 00:15:16:15 - 00:15:19:17 is still OSCAL, but the following process 00:15:19:17 - 00:15:22:17 is fully automated by AI so far, 00:15:22:19 - 00:15:26:22 still C2P enables automating the, process. 00:15:26:22 - 00:15:30:16 We have to, compose the pipelines to invoke C2P 00:15:30:16 - 00:15:34:21 or to deploy policy or whatever. 00:15:35:03 - 00:15:39:00 interaction between, target resources and the compliance frameworks. 00:15:39:05 - 00:15:43:00 But, this can be fully automated by agentic approaches. 00:15:43:05 - 00:15:48:10 So the, final goal is input OSCAL component definition to Agent. Then, 00:15:48:12 - 00:15:53:17 Agent automatically interpret OSCAL's and generate policies and deploy policy 00:15:53:17 - 00:15:55:07 to target resources 00:15:55:07 - 00:15:59:15 and also collect evidence and finally convert to OSCAL assessment results. 00:15:59:15 - 00:16:02:13 These are fully automated and there are no human in the loop. 00:16:02:13 - 00:16:06:01 This is the final goals, and we will, demonstrate and show 00:16:06:05 - 00:16:09:09 how it works on our approach. 00:16:09:15 - 00:16:13:17 Yuji, can I hand over the details about yeah okay. 00:16:13:17 - 00:16:15:15 Please, please stop sharing. 00:16:15:15 - 00:16:19:22 let me explain, how the, CISO compliance is assessment agent work. 00:16:19:22 - 00:16:22:14 typical, compliance assessments, task is, example, to 00:16:22:14 - 00:16:25:22 if we have the new compliance check requirement, so generate 00:16:25:23 - 00:16:29:02 some new policy code and deploy it to the targets 00:16:29:02 - 00:16:33:05 system and enable the automated check and collect evidence from there. 00:16:33:08 - 00:16:37:18 And then the deployed compressed posture this it high level, task. 00:16:37:18 - 00:16:40:08 So we want to automate this, all the testing. 00:16:40:08 - 00:16:43:17 today we need, many, many reports for the enabling this, 00:16:43:22 - 00:16:47:02 so this could be inconsistency and non-compliance and, automate. 00:16:47:04 - 00:16:52:09 So if we enable the automation for this, it's very fixed goal with hard wiring. 00:16:52:12 - 00:16:54:12 it means, rest assured, agility. 00:16:54:12 - 00:16:57:17 And for enabling this automation, we need to bring 00:16:57:17 - 00:17:01:03 the multiple personnel with different knowledge and experience. 00:17:01:03 - 00:17:05:23 So the from the compliance person system administrator, the auditor and CISO. 00:17:06:08 - 00:17:09:02 the knowledge spanning for many areas 00:17:09:02 - 00:17:14:05 So compliance requirement, system architecture, and policy engine, data 00:17:14:08 - 00:17:15:17 And the APIs are available. 00:17:15:17 - 00:17:20:08 So all this information is required for enabling the full compliance automation. 00:17:20:08 - 00:17:22:09 this cannot be done by single person. 00:17:22:09 - 00:17:26:04 So the CISO agent comes in to automate this part. 00:17:26:06 - 00:17:26:24 So okay. 00:17:26:24 - 00:17:28:09 This is a, challenging task. 00:17:28:09 - 00:17:33:02 But eventually if we can enabled automation with aided technology. 00:17:33:02 - 00:17:36:10 so can enable the compliance automation very smoothly. 00:17:36:11 - 00:17:40:06 that's why we are bringing the CISO compliance assessment agent okay. 00:17:40:06 - 00:17:44:05 The idea is CISO compliance agent has many tools. 00:17:44:05 - 00:17:47:13 Policy generation or Policy as a Code Analyst. 00:17:47:13 - 00:17:51:12 and automation tools like, Ansible or any other tool 00:17:51:12 - 00:17:56:00 for automating the target, touching system, GitOps integration, tool 00:17:56:03 - 00:18:00:18 and the C2P, Yana may have already explained and also we have another partner 00:18:00:18 - 00:18:04:12 to agent for accomplishing much higher complex task. 00:18:04:12 - 00:18:09:12 So by using the tools and the agent, we can accomplish 00:18:09:12 - 00:18:14:02 the goal end to end the compliance automation by agentic work 00:18:14:02 - 00:18:17:11 So today, you know, we explained how this, 00:18:17:24 - 00:18:22:00 CISO agent works know, stop sharing, Hiro you take over. 00:18:22:07 - 00:18:24:10 Yes. So, let me share my screen. 00:18:24:10 - 00:18:25:20 Can see characters. 00:18:25:20 - 00:18:30:00 So, the, input of the CISO agent is, high level compliance goal 00:18:30:02 - 00:18:32:17 Text in written in a natural language like this. 00:18:32:17 - 00:18:36:24 and also, there are component definition and also, control. 00:18:36:24 - 00:18:39:24 ID is also specified as a input. 00:18:39:24 - 00:18:44:00 And these three items are input to the CISO agent. 00:18:44:03 - 00:18:47:24 it automatically, dynamically determines the workflow like for example, 00:18:48:02 - 00:18:53:00 based on the control ID, agent will find the, rule ID and rule description. 00:18:53:00 - 00:18:56:07 This means what should it be achieved by the, agent work. 00:18:56:07 - 00:18:59:11 And based on this information, it will generate, policy 00:18:59:11 - 00:19:03:03 code, for example, this is Ansible playbook, or, Kyverno policy 00:19:03:08 - 00:19:04:10 on the Kubernetes cluster. 00:19:04:10 - 00:19:09:04 And generated code will be submitted to, GitHub using the, GitOps 00:19:09:05 - 00:19:13:10 tool described by Yuji for example, if the PR is submitted, 00:19:13:10 - 00:19:17:14 then the, SMEs comes in to the GitHub repository and, 00:19:17:16 - 00:19:22:00 this person will approved a PR And once the PR has been 00:19:22:00 - 00:19:26:00 approved and merged, then the agent will move on to another steps 00:19:26:00 - 00:19:30:10 like for example the execute validated program, Ansible playbook for example. 00:19:30:10 - 00:19:33:18 as a result the policy evaluation result will be generated. 00:19:33:18 - 00:19:36:19 So, it can be automatically generated to assessment result. 00:19:36:19 - 00:19:40:16 This is the CISO agent workflow and or this type of workflows 00:19:40:22 - 00:19:45:01 not a fixed one but it is dynamically, considered by the agent. 00:19:45:16 - 00:19:50:00 And the, actual input and output for CISO agent is, here. 00:19:50:00 - 00:19:54:01 And, example, the component definition Json is something like this. 00:19:54:01 - 00:19:59:03 And here, this has control-ID and this is, another input for CISO agent. 00:19:59:04 - 00:20:03:16 And when the control ID is specified, by checking the component definition 00:20:03:16 - 00:20:08:23 agent will detect in this case, SSH Max startup is the, rule ID. 00:20:09:03 - 00:20:12:09 And also, this is, target requirement result 00:20:12:09 - 00:20:16:07 and it will, generate, some validation program, for example, 00:20:16:14 - 00:20:20:08 Ansible playbook, then after, all the workflow assessment 00:20:20:08 - 00:20:24:00 result for this SSH max startup is generated. 00:20:24:12 - 00:20:27:08 And we already submitted 00:20:27:08 - 00:20:32:20 archive paper for this CISO agent and also another agent Yuji mentioned, 00:20:32:24 - 00:20:36:04 SRE agent, and also the agent framework. 00:20:36:15 - 00:20:37:17 automatically blank. 00:20:37:17 - 00:20:43:02 And also the, the evaluation for this agent, it's automated by our IT bench 00:20:43:14 - 00:20:48:02 capability and, all these our work is submitted as an Archive 00:20:48:02 - 00:20:49:01 Paper and. 00:20:49:01 - 00:20:49:16 Yeah. 00:20:49:16 - 00:20:50:05 Hiro, maybe. 00:20:50:05 - 00:20:53:02 Okay. So you can switch Yeah. Okay. 00:20:53:02 - 00:20:53:23 All right. 00:20:53:23 - 00:20:56:23 Before moving demo, I’d like to briefly 00:20:56:24 - 00:20:59:23 introduce the, evaluation systems. 00:20:59:23 - 00:21:04:02 So creating agent is, we can create agent, 00:21:04:02 - 00:21:09:12 but it's essentially important to evaluate agent. Without evaluations, 00:21:09:12 - 00:21:14:05 We have no way to identify how agent works with or not. 00:21:14:09 - 00:21:19:08 So the evaluation system is, based on my previous scenarios 00:21:19:13 - 00:21:22:13 so I already have, C2P 00:21:22:14 - 00:21:26:14 scenarios to run end to end assessment 00:21:26:23 - 00:21:30:23 against Kubernetes or, Linux machines 00:21:30:23 - 00:21:35:03 using, specific such as, Kyverno or Ansible. 00:21:35:09 - 00:21:40:04 So these, described as a scenarios, for example, in your RHEL case, 00:21:40:04 - 00:21:43:08 we have but non-compliant SSH configuration. 00:21:43:11 - 00:21:47:03 these are described the scenario and also I put set up program 00:21:47:03 - 00:21:53:05 to build RHEL machines and configure such kind of, compressed variation state. 00:21:53:08 - 00:21:58:03 these, compressed in one package we call deployable packages. 00:21:58:11 - 00:22:03:09 Once we created the scenario as a package, then have a framework 00:22:03:09 - 00:22:06:14 Pick the scenario and, deploy the scenario 00:22:06:14 - 00:22:09:14 to create the actual environment. 00:22:09:16 - 00:22:12:10 And configured components variation state. 00:22:12:10 - 00:22:16:09 Once it's ready, then next phase is to invoke 00:22:16:15 - 00:22:20:01 agent with the credentials to access the environment. 00:22:20:05 - 00:22:24:00 And also the goal that, agent should accomplish. 00:22:24:07 - 00:22:29:01 And then that agent framework monitors agent and agent behaviors. 00:22:29:01 - 00:22:33:19 And once the agent say finished, then IT bench framework start evaluations. 00:22:33:19 - 00:22:37:19 The evaluation script is also, composed in each scenario 00:22:37:19 - 00:22:42:22 because it's different between scenario types, scenario and scenarios. 00:22:42:22 - 00:22:48:05 And then finished evaluations, score to the scoreboard, we call it leaderboard. 00:22:48:05 - 00:22:51:03 And then finally, the environment. 00:22:51:03 - 00:22:52:10 This is one cycle. 00:22:52:10 - 00:22:55:24 And we have, a lot of scenarios and, automate 00:22:55:24 - 00:23:00:09 to iterate these cycles to evaluate agent performance. 00:23:00:09 - 00:23:04:22 I created of 50 scenarios based on the CICD benchmarks covering 00:23:04:22 - 00:23:09:08 various, targets, RHEL and the binary Kubernetes. 00:23:09:10 - 00:23:13:20 And also the, scenario complexity is very simple to complex. 00:23:13:24 - 00:23:17:10 It's the simple means just human text 00:23:17:13 - 00:23:20:19 solves the problems, it takes 3 or 4 steps, 00:23:20:23 - 00:23:24:23 but in the complex problems, it takes 6 or 7 steps. 00:23:25:03 - 00:23:30:21 So with these, scenarios, we can evaluate the agent performance. 00:23:30:21 - 00:23:33:21 So this is an example of agent behavior. 00:23:33:23 - 00:23:38:03 So this, left side is the goal in the agent. 00:23:38:05 - 00:23:41:12 So this is the example goal about ensure the cron 00:23:41:12 - 00:23:45:07 configurations in RHEL machine with additional requirements. 00:23:45:07 - 00:23:46:20 This is Control rules. 00:23:46:20 - 00:23:52:10 And to assess this rule, the goal is to create the assessment, programs. 00:23:52:10 - 00:23:58:06 That's, fetch data from RHEL and create checker programs by OPA policies 00:23:58:06 - 00:24:02:09 that compare the gathered status with the expected status. 00:24:02:13 - 00:24:06:08 And after evaluation phase, run this, submit it programs, 00:24:06:08 - 00:24:10:11 whether it can detect the compliance violation or not. 00:24:10:13 - 00:24:16:16 And with these goals the agent starts thinking, what should we do next. So under the agent, 00:24:16:16 - 00:24:21:14 the answer is to generate, playbooks corresponding with these, descriptions. 00:24:21:14 - 00:24:25:16 So agent produces the functions with the arguments. 00:24:25:20 - 00:24:29:08 then the framework picks, run this functions, 00:24:29:14 - 00:24:32:04 then generate and then execute the functions. 00:24:32:04 - 00:24:35:01 The results is, back to the agent. 00:24:35:01 - 00:24:38:19 And the agent will try to consider the next actions. 00:24:38:19 - 00:24:43:17 If the, observation, execution results is, looks good, then, agent 00:24:43:21 - 00:24:49:01 then run the generated playbooks also, the, result is back to the agent. 00:24:49:07 - 00:24:53:09 So this, this, of the observation cycle is iterated 00:24:53:13 - 00:24:57:19 all about, data agent, conclude the program is solved. 00:24:57:19 - 00:24:59:02 The goal is achieved. 00:24:59:02 - 00:25:02:02 Once it's achieved, then the, agents submit. 00:25:02:09 - 00:25:05:06 the fetch programs and check programs 00:25:05:06 - 00:25:09:02 to the frame ID fetch servers then evaluation, study. 00:25:09:02 - 00:25:11:13 This is the example agent behavior. 00:25:11:13 - 00:25:14:13 this behavior depends on the, based models. 00:25:14:13 - 00:25:19:03 and large model good performance, but, small model, not. 00:25:19:10 - 00:25:25:03 So these kind of, evaluation can be done via this IT bench automation. 00:25:25:09 - 00:25:25:19 Okay. 00:25:25:19 - 00:25:30:02 Hiro will show concrete demo of these, so, Hiro, you can show? 00:25:30:09 - 00:25:32:23 Yes. So, let me share my screen. Okay. 00:25:32:23 - 00:25:33:06 Okay. 00:25:33:06 - 00:25:38:01 So, as I, described earlier, the agent input is, high level 00:25:38:01 - 00:25:43:06 compliance goal between natural language and in this example, there is a 00:25:43:13 - 00:25:46:24 compress requirement coming from, some compliance standards 00:25:47:03 - 00:25:50:23 and also, the, goal description has some, detailed information 00:25:50:23 - 00:25:56:15 for what should be done by the agent. Agent already knows its available tools 00:25:56:15 - 00:26:01:05 in this example, policy generation and the policy deployment are required. 00:26:01:05 - 00:26:04:10 And in such case, in this example, 00:26:04:10 - 00:26:07:24 the agent is force checking the existing policy because, 00:26:08:09 - 00:26:12:24 if there is any, existing policy must be, compared to the, goal description. 00:26:12:24 - 00:26:18:05 And if the goal is already satisfied, then, agent can quit its work. 00:26:18:05 - 00:26:21:23 But in this example, there is no existing policy on the cluster, 00:26:22:01 - 00:26:27:11 so, it will try to generate, new policy by using, policy generation tool. And 00:26:29:00 - 00:26:30:20 once it is generated, it will 00:26:30:20 - 00:26:34:13 try to deploy the generated policy to the specified cluster. 00:26:35:07 - 00:26:38:22 And after that, not only generation deployment, 00:26:38:22 - 00:26:42:00 but also, it is confirming the deployment, because, 00:26:42:04 - 00:26:46:02 if there is any error on the deployment, it should be redo. 00:26:46:06 - 00:26:51:11 And Now, it confirmed the deployment, so CISO 00:26:51:11 - 00:26:52:08 agent finished. 00:26:52:08 - 00:26:56:06 So, what I would like to emphasize here is, all of the workflow is dynamically 00:26:56:12 - 00:26:57:22 Done by the agent. 00:26:57:22 - 00:26:59:22 And if there is any errors on 00:26:59:22 - 00:27:03:15 the generated policy and the deployment was not successful, 00:27:03:17 - 00:27:07:02 the CISO agent will regenerate policy and retry 00:27:07:02 - 00:27:10:02 the deployment. 00:27:10:02 - 00:27:14:14 And now in this demo, I am checking the, policy agent 00:27:14:14 - 00:27:18:02 work result manually and in this example, 00:27:18:07 - 00:27:21:07 the generated policy is actually deployed and Kyverno 00:27:21:09 - 00:27:25:17 policy engine on the cluster is actually reporting the compliance issue 00:27:25:21 - 00:27:28:20 detected by the policy, which the agent generated 00:27:28:20 - 00:27:32:13 So, there are some missing GitOps interactions and, 00:27:32:18 - 00:27:34:04 assessment result generation. 00:27:34:04 - 00:27:37:13 But all those can be also done by the agent itself. 00:27:37:15 - 00:27:41:15 So, for, the conversion from control component definition 00:27:41:22 - 00:27:45:18 to assessment result, or automated by the CISO agent. 00:27:46:02 - 00:27:47:02 Yeah. 00:27:47:02 - 00:27:52:01 finally to all the component present it today is basically the open source, 00:27:52:05 - 00:27:55:13 agent and, sample scenario, automation is, GitHub. 00:27:55:13 - 00:27:58:24 So, you can try this CISO agent, on your machine 00:27:59:14 - 00:28:02:09 then we, preparing the, benchmark leaderboard right now. 00:28:02:09 - 00:28:04:17 So, it will be available sometime very soon. 00:28:04:17 - 00:28:05:17 Okay. Thank you so much. 00:28:05:17 - 00:28:08:16 Any questions? open to the questions. So thank you so much. 00:28:08:16 - 00:28:09:08 Thank you. 00:28:09:08 - 00:28:10:16 Yuji-san, 00:28:10:16 - 00:28:12:09 And Hiro-san, and Yana-san. 00:28:12:09 - 00:28:15:01 Just one comment on the scenarios. 00:28:15:01 - 00:28:17:07 Open source repo that was shared. 00:28:17:07 - 00:28:19:13 You are going to find, four scenarios. 00:28:19:13 - 00:28:23:12 The reason why we don't open source, all the 50 scenarios that we have in 00:28:23:12 - 00:28:26:16 the bench is because we don't want the people registering 00:28:26:16 - 00:28:29:24 their agents to over feed them for the scenario. 00:28:29:24 - 00:28:31:07 So they are kept private 00:28:31:07 - 00:28:35:16 for the benefit of having, proper, benchmarking of the, of the agents.