Overview of source types for Data Manager
You can use Data Manager to ingest data of the following source types.
To request access to the Amazon S3 data source, select Amazon Web Services as a data input to onboard, and Amazon S3 as a data source, and select Connect to Splunk Cloud Platform.
After you install the add-on that is applicable to your data input type, Data Manager supports Common Information Model (CIM) normalization. Installing the add-on enables data processing that occurs when a search is run, called search-time processing. You must install the add-on on the part of your Splunk Cloud deployment that performs the parsing or search-time functionality for your data. You don't need to configure the add-on.
For more information on which add-on applies to your data input type, see the Prerequisites topic in the chapter of your cloud data input type in this manual.
For information on the CIM, see the Overview of the Splunk Common Information Model topic in the Common Information Model Add-on manual.
Getting data in for AWS
You can get data in for the following AWS data sources.
Data source | Description | Source type and example event |
---|---|---|
Amazon API Gateway | Amazon API Gateway is an AWS service for creating, publishing, maintaining, monitoring, and securing REST, HTTP, and WebSocket APIs at any scale. Amazon API Gateway reports metrics through Amazon CloudWatch. | Source type:
Example event: { [-] additionalEventData: { [+] } awsRegion: us-east-1 eventCategory: Management eventID: a41227dd-c5d0-45f3-8fb3-40fba334a6ef eventName: GetBucketAcl eventSource: s3.amazonaws.com eventTime: 2021-05-12T23:09:48Z eventType: AwsApiCall eventVersion: 1.08 managementEvent: true readOnly: true recipientAccountId: 486996137179 requestID: B9V4WAKPAR6Q0M20 requestParameters: { [+] } resources: [ [+] ] responseElements: null sharedEventID: 0b5a110a-4364-461f-8472-709e2cc67a8f sourceIPAddress: cloudtrail.amazonaws.com userAgent: cloudtrail.amazonaws.com userIdentity: { [+] } } |
AWS CloudHSM | AWS CloudHSM is a cloud-based hardware security module (HSM) that lets you generate and use your own encryption keys on the AWS Cloud. | Source type:
Example event: { [-] additionalEventData: { [+] } awsRegion: us-east-1 eventCategory: Management eventID: a41227dd-c5d0-45f3-8fb3-40fba334a6ef eventName: GetBucketAcl eventSource: s3.amazonaws.com eventTime: 2021-05-12T23:09:48Z eventType: AwsApiCall eventVersion: 1.08 managementEvent: true readOnly: true recipientAccountId: 486996137179 requestID: B9V4WAKPAR6Q0M20 requestParameters: { [+] } resources: [ [+] ] responseElements: null sharedEventID: 0b5a110a-4364-461f-8472-709e2cc67a8f sourceIPAddress: cloudtrail.amazonaws.com userAgent: cloudtrail.amazonaws.com userIdentity: { [+] } } |
AWS CloudTrail | AWS CloudTrail is a service that enables governance, compliance, operational auditing, and risk auditing of your AWS account. | Source type:
Example event: { [-] additionalEventData: { [+] } awsRegion: us-east-1 eventCategory: Management eventID: a41227dd-c5d0-45f3-8fb3-40fba334a6ef eventName: GetBucketAcl eventSource: s3.amazonaws.com eventTime: 2021-05-12T23:09:48Z eventType: AwsApiCall eventVersion: 1.08 managementEvent: true readOnly: true recipientAccountId: 486996137179 requestID: B9V4WAKPAR6Q0M20 requestParameters: { [+] } resources: [ [+] ] responseElements: null sharedEventID: 0b5a110a-4364-461f-8472-709e2cc67a8f sourceIPAddress: cloudtrail.amazonaws.com userAgent: cloudtrail.amazonaws.com userIdentity: { [+] } } |
Amazon DocumentDB | Amazon DocumentDB (with MongoDB compatibility) is a fully managed database service that is purpose-built for JSON data management at scale. | Source type:
Example event: { [-] additionalEventData: { [+] } awsRegion: us-east-1 eventCategory: Management eventID: a41227dd-c5d0-45f3-8fb3-40fba334a6ef eventName: GetBucketAcl eventSource: s3.amazonaws.com eventTime: 2021-05-12T23:09:48Z eventType: AwsApiCall eventVersion: 1.08 managementEvent: true readOnly: true recipientAccountId: 486996137179 requestID: B9V4WAKPAR6Q0M20 requestParameters: { [+] } resources: [ [+] ] responseElements: null sharedEventID: 0b5a110a-4364-461f-8472-709e2cc67a8f sourceIPAddress: cloudtrail.amazonaws.com userAgent: cloudtrail.amazonaws.com userIdentity: { [+] } } |
Amazon EKS | Amazon EKS is a managed service that you can use to run Kubernetes on AWS without installing, operating, and maintaining your own Kubernetes control plane or nodes. | Source type:
Example event: { [-] additionalEventData: { [+] } awsRegion: us-east-1 eventCategory: Management eventID: a41227dd-c5d0-45f3-8fb3-40fba334a6ef eventName: GetBucketAcl eventSource: s3.amazonaws.com eventTime: 2021-05-12T23:09:48Z eventType: AwsApiCall eventVersion: 1.08 managementEvent: true readOnly: true recipientAccountId: 486996137179 requestID: B9V4WAKPAR6Q0M20 requestParameters: { [+] } resources: [ [+] ] responseElements: null sharedEventID: 0b5a110a-4364-461f-8472-709e2cc67a8f sourceIPAddress: cloudtrail.amazonaws.com userAgent: cloudtrail.amazonaws.com userIdentity: { [+] } } |
Amazon GuardDuty | Amazon GuardDuty is a threat detection service that continuously monitors for malicious activity and unauthorized behavior to protect your AWS accounts, workloads, and data stored in Amazon S3. | Source type:
Example event: { [-] accountId: 467463828374 arn: arn:aws:guardduty:us-west-2:467463828374:detector/66b9e579d125e7ca30e97cd350f3cc06/finding/28b9e580b2e3aafe7570484bcdf11b12 createdAt: 2021-05-12T19:41:31.000000Z description: API ListStacks was invoked using root credentials from IP address 91.10.46.14. id: 28b9e580b2e3aafe7570484bcdf11b12 partition: aws region: us-west-2 resource: { [+] } schemaVersion: 2.0 service: { [+] } severity: 3 title: API ListStacks was invoked using root credentials. type: Policy:IAMUser/RootCredentialUsage updatedAt: 2021-05-12T19:41:31.000000Z } |
AWS IAM Access Analyzer | AWS IAM Access Analyzer helps you identify the resources in your organization and accounts, such as Amazon S3 buckets or IAM roles, that are shared with an external entity. | Source type:
Example event: { [-] accountId: 55000000000 action: [ [+] ] analyzedAt: 2021-02-04T17:21:23.130Z condition: { [+] } createdAt: 2021-02-04T17:21:23.130Z id: 723cc4fd-97bc-43b8-8932-6889c4070e0e isDeleted: false isPublic: false principal: { [+] } region: us-west-2 resource: arn:aws:iam::55000000000:role/SplunkDMStackSetExecutionRole resourceType: AWS::IAM::Role status: ACTIVE updatedAt: 2021-02-04T17:21:23.130Z version: 1.0 } |
AWS IAM Credential Report | AWS IAM Credential Report lists all users in your account and the status of their various credentials, including passwords, access keys, and MFA devices. | Source type:
Example event: { [-] access_key_1_active: false access_key_1_last_rotated: N/A access_key_1_last_used_date: N/A access_key_1_last_used_region: N/A access_key_1_last_used_service: N/A access_key_2_active: false access_key_2_last_rotated: N/A access_key_2_last_used_date: N/A access_key_2_last_used_region: N/A access_key_2_last_used_service: N/A account_id: 45000000000 arn: arn:aws:iam::45000000000:root cert_1_active: false cert_1_last_rotated: N/A cert_2_active: false cert_2_last_rotated: N/A SplunkDM_aws_configid: d8b11c12-6707-11eb-95ab-02cbcd7b93b9 mfa_active: true password_enabled: not_supported password_last_changed: not_supported password_last_used: 2020-12-09T21:15:59+00:00 password_next_rotation: not_supported user: <root_account> user_creation_time: 2020-09-12T15:48:42+00:00 } |
AWS Lambda | AWS Lambda is a compute service that lets you run code without provisioning or managing servers. | Source type:
Example event: { [-] additionalEventData: { [+] } awsRegion: us-east-1 eventCategory: Management eventID: a41227dd-c5d0-45f3-8fb3-40fba334a6ef eventName: GetBucketAcl eventSource: s3.amazonaws.com eventTime: 2021-05-12T23:09:48Z eventType: AwsApiCall eventVersion: 1.08 managementEvent: true readOnly: true recipientAccountId: 486996137179 requestID: B9V4WAKPAR6Q0M20 requestParameters: { [+] } resources: [ [+] ] responseElements: null sharedEventID: 0b5a110a-4364-461f-8472-709e2cc67a8f sourceIPAddress: cloudtrail.amazonaws.com userAgent: cloudtrail.amazonaws.com userIdentity: { [+] } } |
Metadata | Metadata is data about your instance that you can use to configure or manage items such as EC2 Instances, IAM Roles, and Security Groups. | Source type:
Example event: { [-] AccountId: 486996137179 Associations: [ [+] ] Entries: [ [+] ] IsDefault: true NetworkAclId: acl-ccf682a7 OwnerId: 486996137179 Region: us-east-2 Tags: [ [+] ] VpcId: vpc-9cb73ef7 SplunkDM_input_id: f992eedc-e815-4eaf-998f-894a994719ac } |
Amazon RDS | Amazon RDS is a web service that allows users to set up, operate, and scale a relational database in the cloud. | Source type:
Example event: { [-] additionalEventData: { [+] } awsRegion: us-east-1 eventCategory: Management eventID: a41227dd-c5d0-45f3-8fb3-40fba334a6ef eventName: GetBucketAcl eventSource: s3.amazonaws.com eventTime: 2021-05-12T23:09:48Z eventType: AwsApiCall eventVersion: 1.08 managementEvent: true readOnly: true recipientAccountId: 486996137179 requestID: B9V4WAKPAR6Q0M20 requestParameters: { [+] } resources: [ [+] ] responseElements: null sharedEventID: 0b5a110a-4364-461f-8472-709e2cc67a8f sourceIPAddress: cloudtrail.amazonaws.com userAgent: cloudtrail.amazonaws.com userIdentity: { [+] } } |
AWS Security Hub | AWS Security Hub gives you a comprehensive view of your security alerts and security posture across your AWS accounts. | Source type:
Example event: { [-] AwsAccountId: 986546787665 CreatedAt: 2021-05-12T19:40:31.000000Z Description: API ListStacks was invoked using root credentials from IP address 99.37.245.87. FirstObservedAt: 2021-05-12T19:40:31Z GeneratorId: arn:aws:guardduty:us-west-1:264962456697:detector/66b954920ec49f3fb2b48aac9f4dfe55 Id: arn:aws:guardduty:us-west-1:264962456697:detector/66b954920ec49f3fb2b48aac9f4dfe55/finding/48b9761c3553de180eeae02247363b8d LastObservedAt: 2021-05-12T19:40:31Z ProductArn: arn:aws:securityhub:us-west-1::product/aws/guardduty ProductFields: { [+] } RecordState: ACTIVE Resources: [ [+] ] SchemaVersion: 2018-10-08 Severity: { [+] } SourceUrl: https://us-west-1.console.aws.amazon.com/guardduty/home?region=us-west-1#/findings?macros=current&fId=48b9761c3553de180eeae02247363b8d Title: API ListStacks was invoked using root credentials. Types: [ [+] ] UpdatedAt: 2021-05-12T19:40:31.000000Z Workflow: { [+] } WorkflowState: NEW } |
AWS CloudTrail (S3) | Gain visibility into CloudTrail metrics from their S3 logs. | Source type:
Example event: { "eventVersion": "1.03", "userIdentity": { "type": "IAMUser", "principalId": "111122223333", "arn": "arn:aws:iam::111122223333:user/myUserName", "accountId": "111122223333", "accessKeyId": "AKIAIOSFODNN7EXAMPLE", "userName": "myUserName" }, "eventTime": "2019-02-01T03:18:19Z", "eventSource": "s3.amazonaws.com", "eventName": "ListBuckets", "awsRegion": "us-west-2", "sourceIPAddress": "127.0.0.1", "userAgent": "[]", "requestParameters": { "host": [ "s3.us-west-2.amazonaws.com" ] }, "responseElements": null, "additionalEventData": { "SignatureVersion": "SigV2", "AuthenticationMethod": "QueryString" }, "requestID": "47B8E8D397DCE7A6", "eventID": "cdc4b7ed-e171-4cef-975a-ad829d4123e8", "eventType": "AwsApiCall", "recipientAccountId": "111122223333" } |
AWS S3 access logs (S3) | Gain visibility into S3 access logs from their S3 logs. | Source type:
Example event: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx testbucket [04/Feb/2021:17:53:36 +0000] 98.33.33.216 - xxxxxxxxxxxx REST.DELETE.OBJECT xxxxxxxxxxxx.txt "DELETE /xxxxxxxxxxxx.txt HTTP/1.1" 400 AuthorizationHeaderMalformed 365 - 3 - "-" "Boto3/1.9.91 Python/3.8.5 Linux/5.4.0-65-generic Botocore/1.12.253" - xxxx/xxxxxxxxxxxx= SigV4 ECDHE-RSA-AES128-GCM-SHA256 AuthHeader testbucket.s3.amazonaws.com TLSv1.2 |
AWS Load Balancer (ELB) access logs | Gain visibility into ELB access logs from their S3 logs. | Source type:
Example event: http 2021-02-05T01:44:04.252695Z app/appelb2/xxxxxxxxxxxx xxx.xx.xx.xxx:xxxx xxx.xx.xx.x:xx 0.000 0.001 0.000 200 200 348 271 "GET http://appelb2-1376785808.us-east-2.elb.amazonaws.com:80/ HTTP/1.1" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" - - arn:aws:elasticloadbalancing:us-east-2:xxxxxxxxxxxx:targetgroup/instance/xxxxxxxxxxxx "Root=1-601ca2e4-xxxxxxxxxxxxxxxx" "-" "-" 0 2021-02-05T01:44:04.251000Z "forward" "-" "-" "xxx.xx.xx.x:xx" "200" "-" "-" |
AWS CloudFront (CF) access logs | Gain visibility into CloudFront access logs from their S3 logs. | Source type:
Example event: 2021-02-05 03:21:02 SFO53-C1 946 xxx:xxx:xxxx:xxxx:xxx:xxxx:xxxx:xxxx GET xxxxxxxxxxxxxxx.cloudfront.net / 307 - Mozilla/5.0%20(X11;%20Linux%20x86_64)%20AppleWebKit/537.36%20(KHTML,%20like%20Gecko)%20Chrome/81.0.4044.138%20Safari/537.36 - - Miss xSsEXRrrvec1WehsbGWX9C8CwP26AX3HLxwC-MO62C9leZShcMRgrQ== xxxxxxxxxxxxxxx.cloudfront.net http 472 0.074 - - - Miss HTTP/1.1 - - 52612 0.074 Miss application/xml - - - |
Getting data in for Microsoft Azure
You can get data in for the following Microsoft Azure data sources.
Data source | Description | Source type and example event |
---|---|---|
Microsoft Entra ID | Microsoft Entra ID is a cloud computing service created by Microsoft for building, testing, deploying, and managing applications and services through Microsoft-managed data centers. | Source type:
Example event: { [-] Level: 4 callerIpAddress: 52.43.55.129 category: ServicePrincipalSignInLogs correlationId: 17b0805a-13f0-4800-a81a-d1ea2d1a9921 data_manager_input_id: 089a37ba-59f3-450a-9201-e8aa9032027e durationMs: 0 location: US operationName: Sign-in activity operationVersion: 1.0 properties: { [+] } resourceId: /tenants/501792f2-ef2c-4251-957b-293fadb63ddc/providers/Microsoft.aadiam resultSignature: None resultType: 0 tenantId: 501792f2-ef2c-4251-957b-293fadb63ddc time: 2021-09-18T18:00:09.4379696Z |
Azure Activity Logs | Azure Activity Logs are platform logs in Azure that provide insight into subscription-level events. This includes such information as when a resource is modified or when a virtual machine is started. | Source type:
Example event: { [-] ReleaseVersion: 6.2021.41.6+f1cf8a2.release_2021w41 RoleLocation: East US callerIpAddress: 20.42.74.11 category: Administrative correlationId: 804f69ae-aedb-499f-9e13-1157121456b4 durationMs: 69 identity: { [+] } level: Information operationName: MICROSOFT.STORAGE/STORAGEACCOUNTS/LISTACCOUNTSAS/ACTION properties: { [+] } resourceId: /SUBSCRIPTIONS/C83C2282-2E21-4F64-86AE-FDFA66B673EB/RESOURCEGROUPS/SIMON-DEMO/PROVIDERS/MICROSOFT.STORAGE/STORAGEACCOUNTS/DSPCDCSIMONTEST resultSignature: Succeeded.OK resultType: Success tenantId: 501792f2-ef2c-4251-957b-293fadb63ddc time: 2021-10-21T17:36:11.2611638Z } |
Azure Event Hub | Available for private preview customers. Azure Event Hubs are data streaming services that can process large volumes of events per second with low latency. They can stream events from any source to any destination. |
Source types:
Example event: { [-] Level: 4 callerIpAddress: 52.43.55.129 category: ServicePrincipalSignInLogs correlationId: 17b0805a-13f0-4800-a81a-d1ea2d1a9921 data_manager_input_id: 089a37ba-59f3-450a-9201-e8aa9032027e durationMs: 0 location: US operationName: Sign-in activity operationVersion: 1.0 properties: { [+] } resourceId: /tenants/501792f2-ef2c-4251-957b-293fadb63ddc/providers/Microsoft.aadiam resultSignature: None resultType: 0 tenantId: 501792f2-ef2c-4251-957b-293fadb63ddc time: 2021-09-18T18:00:09.4379696Z |
Azure Event Hubs
The Azure Event Hub data input is available for private preview customers. If you'd like to participate in the Splunk private preview program, contact your account team.
Using Data Manager, you can ingest data from Azure Event Hubs to Splunk Cloud Platform. The Azure Event Hubs functionality provides an event streaming platform that you can scale. It can process large volumes of data units, called events, per second. To learn about Azure Event Hubs and their throughput capacity, see the following articles in the Azure documentation:
Event Hubs reside in a management container called a namespace. It can contain one or more event hubs. When you create an event hub on Azure, you need to select an existing namespace or create one.
The entities that send data to an event hub are event publishers, also called event producers. Event consumers are entities that read data from the event hub, like Splunk Cloud Platform. You can divide event consumers into consumer groups. Although a consumer group can include several event consumers, the best practice is to create a separate consumer group for each.
By creating partitions on an event hub, you can organize events and help processing large volumes of events. Partitions multiply the available capacity and scale out the number of parallel consumers. Also, you can map events to partitions, so that each event has a processing owner. To learn about features of Event Hubs, see Features and terminology in Azure Event Hubs in the Azure documentation.
Partition count is the total number of partitions in an event hub. It must be between one and the maximum limit which depends on the event hub tier. The best practice is to choose the number of partitions that you expect to need during the peak load on an event hub.
After creating an event hub, you can increase a partition count for the premium and dedicated tiers. For other tiers, you can't change the partition count for an event hub.
The event lives on the event hub for a limited time that varies from 24 hrs to 7 days. It is called a retention time and depends on the event hub tier. To learn about tiers of Event Hubs, see: Basic vs. standard vs. premium vs. dedicated tiers in the Azure documentation.
Getting data in for Google Cloud Platform
You can get data in for the following Google Cloud Platform data sources.
Data source | Description | Source type and example event |
---|---|---|
Audit Logs - Admin Activity | Audit Logs - Admin Activity records administrative activities within your Google Cloud resources. | Source type:
{ "protoPayload": { "@type": "type.googleapis.com/google.cloud.audit.AuditLog", "authenticationInfo": { "principalEmail": "system:kube-controller-manager" }, "authorizationInfo": [ { "granted": true, "permission": "io.k8s.coordination.v1.leases.update", "resource": "coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager" } ], "methodName": "io.k8s.coordination.v1.leases.update", "requestMetadata": { "callerIp": "::1", "callerSuppliedUserAgent": "kube-controller-manager/v1.19.10 (linux/amd64) kubernetes/xxxxxx/leader-election" }, "resourceName": "coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager", "serviceName": "k8s.io", "status": { "code": 0 } }, "insertId": "ffffffff-31b8-4291-b8bd-30dc6ab7f6a8", "resource": { "type": "k8s_cluster", "labels": { "cluster_name": "cal2z-example", "location": "us-west1-a", "project_id": "dev-example" } }, "timestamp": "2021-06-24T17:28:09.822777Z", "labels": { "authorization.k8s.io/reason": "RBAC: allowed by ClusterRoleBinding 'system:kube-controller-manager' of ClusterRole 'system:kube-controller-manager' to User 'system:kube-controller-manager'", "authorization.k8s.io/decision": "allow" }, "logName": "projects/dev-example/logs/cloudaudit.googleapis.com%2Factivity", "operation": { "id": "ffffffff-31b8-4291-b8bd-30dc6ab7f6a8", "producer": "k8s.io", "first": true, "last": true }, "receiveTimestamp": "2021-06-24T17:28:11.494703030Z" } |
Audit Logs - System Events | Audit Logs - System Events records administrative activities within your Google Cloud resources. | Source type:
{ "protoPayload": { "@type": "type.googleapis.com/google.cloud.audit.AuditLog", "serviceName": "pubsub.googleapis.com", "methodName": "Subscriber.InternalExpireInactiveSubscription", "resourceName": "projects/123456789101/subscriptions/splunk_example-7d26e38e2e7d_8a7df49a6fc708a33cb5def0fffeeed6b22bd5699d50a1afb2ce891663cc34" }, "insertId": "suv6x4c3ho", "resource": { "type": "pubsub_subscription", "labels": { "subscription_id": "projects/123456789101/subscriptions/splunk_example-7d26e38e2e7d_8a7df49a6fc708a33cb5def0fffeeed6b22bd5699d50a1afb2ce891663cc34", "project_id": "dev-example" } }, "timestamp": "2021-06-24T09:59:13.553133776Z", "severity": "INFO", "logName": "projects/dev-example/logs/cloudaudit.googleapis.com%2Fsystem_event", "receiveTimestamp": "2021-06-24T09:59:14.440519798Z" } |
Audit Logs - Policy Denied | Audit Logs - Policy Denied records administrative activities within your Google Cloud resources. | Source type:
{ "insertId": "1234ljc6f7", "logName": "projects/corp-storage/logs/cloudaudit.googleapis.com%2Fpolicy", "protoPayload": { "@type": "type.googleapis.com/google.cloud.audit.AuditLog", "authenticationInfo": { "principalEmail": "someone@google.com" }, "metadata": { "@type": "type.googleapis.com/google.cloud.audit.VpcServiceControlAuditMetadata", "resourceNames": [ { "0": "projects/_" } ], "violationReason": "NO_MATCHING_ACCESS_LEVEL" }, "methodName": "google.storage.NoBillingOk", "requestMetadata": { "callerIp": "xxxx:xxxx:xxxx:xxxx:d358:586b:db59:9617", "destinationAttributes": {}, "requestAttributes": {} }, "resourceName": "projects/987654321012", "serviceName": "storage.googleapis.com", "status": { "code": 7, "details": [ { "0": { "@type": "type.googleapis.com/google.rpc.PreconditionFailure", "violations": [ { "0": { "type": "VPC_SERVICE_CONTROLS" } } ] } } ], "message": "Request is prohibited by organization's policy" } }, "receiveTimestamp": "2018-11-27T21:40:43.823209571Z", "resource": { "labels": { "method": "google.storage.NoBillingOk", "project_id": "corp-storage", "service": "storage.googleapis.com" }, "type": "audited_resource" }, "severity": "ERROR", "timestamp": "2018-11-27T21:40:42.973784140Z" } |
Audit Logs - Data Access | Audit Logs - Data Access records administrative activities within your Google Cloud resources. | Source type:
Example event:
{ "protoPayload": { "@type": "type.googleapis.com/google.cloud.audit.AuditLog", "authenticationInfo": { "principalEmail": "109633456012-compute@developer.example.com", "serviceAccountDelegationInfo": [ { "firstPartyPrincipal": { "principalEmail": "service-109633456012-compute@developer.example.com" } } ] }, "requestMetadata": { "callerIp": "xx.xxx.xx.xx", "callerSuppliedUserAgent": "opentelemetry-collector-contrib grpc-go/1.36.1,gzip(gfe)", "callerNetwork": "//compute.googleapis.com/projects/dev-example/global/networks/__unknown__", "requestAttributes": { "time": "2021-06-28T22:43:18.758057304Z", "auth": {} }, "destinationAttributes": {} }, "serviceName": "monitoring.googleapis.com", "methodName": "google.monitoring.v3.MetricService.CreateTimeSeries", "authorizationInfo": [ { "resource": "109633456012", "permission": "monitoring.timeSeries.create", "granted": true, "resourceAttributes": {} } ], "resourceName": "projects/dev-example", "request": { "@type": "type.googleapis.com/google.monitoring.v3.CreateTimeSeriesRequest", "name": "projects/dev-example" } }, "insertId": "1lq2cbckjjo123xt0", "resource": { "type": "audited_resource", "labels": { "method": "google.monitoring.v3.MetricService.CreateTimeSeries", "project_id": "dev-example", "service": "monitoring.googleapis.com" } }, "timestamp": "2021-06-28T22:43:18.754427398Z", "severity": "INFO", "logName": "projects/dev-example/logs/cloudaudit.googleapis.com%2Fdata_access", "receiveTimestamp": "2021-06-28T22:43:19.714577654Z" } |
Access Transparency Logs - Access Transparency | Review logs of actions taken by Google staff when accessing user content. User-generated content is text entered into Gmail, Docs, Sheets, Slides, and other apps. | Source type:
Example event: { "insertId": "abcdefg12345", "jsonPayload": { "@type": "type.googleapis.com/google.cloud.audit.TransparencyLog", "location": { "principalOfficeCountry": "US", "principalEmployingEntity": "Google LLC", "principalPhysicalLocationCountry": "CA" }, "product": [ { "0": "Cloud Storage" } ], "reason": [ { "detail": "Case number: bar123", "type": "CUSTOMER_INITIATED_SUPPORT" } ], "accesses": [ { "0": { "methodName": "GoogleInternal.Read", "resourceName": "//googleapis.com/storage/buckets/BUCKET_NAME/objects/foo123" } } ], "accessApprovals": [ { "0": "projects/123/approvalRequests/abcdef12345" } ] }, "logName": "projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Faccess_transparency", "operation": { "id": "12345xyz" }, "receiveTimestamp": "2021-06-28T22:43:19.714577654Z", "resource": { "labels": { "project_id": "1234567890" }, "type": "project" }, "severity": "NOTICE", "timestamp": "2021-06-28T22:43:19.714577654Z" } |
Set up Data Manager | Data ingestion mechanisms and intervals in Data Manager |
This documentation applies to the following versions of Data Manager: 1.10.0
Feedback submitted, thanks!