Event-Based Workflows: The Concept and Need

Can you explain what inspired the development of Event-Based Workflows? What specific gaps in insider threat detection are we addressing with this feature?

The development of Event-Based Workflows was driven by the need to address dynamic and context-sensitive risks within modern SaaS environments. While our original Workflows are highly effective at monitoring and flagging assets based on specific content or configurations, there is a key gap: organisations need to respond to actions and behaviours that may indicate insider threats or unintended activities, rather than relying solely on static data patterns.

For instance, actions such as sharing a sensitive file publicly, downloading an unusual volume of documents, or deleting critical assets could signal oversharing, data exfiltration, or even malicious intent. These behaviours require swift responses to mitigate risks effectively. Event-Based Workflows fill this gap by enabling organisations to detect such activities in real time and take immediate action - whether by notifying the security team or suspending the user. This approach shifts the focus from reactive data monitoring to proactive behaviour-driven risk management.

What are the most common insider threat scenarios that you and your team have encountered in research or customer feedback that helped shape the feature?

Common insider threat scenarios that shaped Event-Based Workflows include data exfiltration, where employees download large volumes of files before leaving or sharing them externally; data oversharing, such as mistakenly sharing confidential files outside the organisation;  and data sabotage, like unauthorised file deletion. These risks showed us how important it is to spot these behaviours in real time and act quickly with alerts, user suspension or any other appropriate action.

Technical Foundations

Can you walk us through the technical solution powering Event-Based Workflows? How does it integrate with existing cloud collaboration platforms like Google Workspace?

Metomic ingests events from the APIs of SaaS platforms like Google Workspace. These events are evaluated against configured workflows, and if the observed events match the workflow conditions, they are triggered to perform the configured actions, such as sending notifications, suspending a user etc. The specific events and how we fetch them depend on the available API - it could be us listening to webhooks or polling endpoints for recent activities. For instance, integration with Google Drive's API allows Metomic to detect events like file downloads, deletions, or external sharing.

What are the key technical challenges when designing a feature that can detect and respond to suspicious data activity in real time?

One of the key challenges is managing the high volume of data generated over time. Storing all events isn't practical due to sheer scale, so we must carefully balance retention policies, data aggregation, and event filtering to ensure we're capturing meaningful signals without overwhelming storage capacity.

Another challenge is real-time processing. Low latency is critical - delays could mean failing to prevent a security incident. To achieve this, our data ingestion and processing pipelines must be highly optimised for efficiency, with the capacity to handle high throughput under heavy loads. This requires a resilient architecture capable of scaling horizontally to manage spikes in activity without causing bottlenecks or data pile-ups. These considerations ensure the system remains responsive and reliable, even in high-demand scenarios.

Response Mechanisms and Automation

The feature includes automated responses like suspending user activity. What was the rationale behind choosing these specific responses, and how do they balance security with user experience?

Event-Based Workflows support various configurable actions. For instance, some organisations may opt for no immediate action, choosing instead to let security teams manually investigate incidents. Others might prefer sending alerts or notifications directly to users, offering an opportunity to correct mistakes or clarify their actions without escalating further.

Suspending user activity is included for high-risk scenarios, where swift intervention is critical to prevent data loss or breaches. This approach is particularly suited for risk-averse environments, as it acts as a precautionary measure while the security team investigates the incident.

How customisable are these workflows for different organisations, and what considerations should security teams keep in mind when configuring them?

Event-Based Workflows are highly customisable, giving organisations the flexibility to tailor them to their unique security needs and operational workflows. Security teams can define specific conditions for triggering a workflow, such as detecting sensitive files shared publicly or unusual download patterns, and configure the responses that best suit their environment. These responses range from taking no immediate action, to notifying administrators or users, to triggering a webhook for an external system integration like SIEM.

When configuring workflows, security teams should consider the balance between automation and oversight. For example, automated responses like suspending user activity are ideal for high-risk scenarios but might be excessive for less critical events. Similarly, webhooks allow seamless integration into existing tools, enabling more comprehensive incident management without disrupting existing processes. Ultimately, configurations should align with the organisation's risk tolerance and ensure that the workflows are both effective and minimally disruptive to regular operations.

Performance and Scalability

Detecting large-scale downloads or suspicious activity in real time must require significant performance optimisation. Can you share how our team ensures scalability for large organisations with thousands of employees?

Indeed, ensuring scalability for large organisations with thousands of employees requires a robust and optimised architecture. We employ a distributed and horizontally scalable system for event ingestion and processing, so we can dynamically scale resources to handle spikes in activity without compromising performance or introducing latency.

To optimise performance, we implement streaming data pipelines that process events in real time rather than batch, ensuring low-latency detection. Efficient indexing and filtering allow us to process only relevant events, reducing computational overhead. This approach ensures that we can detect large-scale downloads or other suspicious activities promptly even for large organisations.

Security and Privacy Considerations

With a feature that monitors data activity, privacy is a major concern. How does Metomic ensure that we’re maintaining user privacy while still being effective in threat detection?

Metomic is committed to safeguarding user privacy while effectively detecting threats. We adhere to data minimisation principles, collecting only the data necessary for security purposes and avoiding excessive or irrelevant information. To protect sensitive information, we employ data masking and redaction techniques, ensuring that personal data remains confidential during processing. By integrating these measures, Metomic ensures that user privacy is maintained without compromising the effectiveness of threat detection.

What measures are in place to secure the data we process and store as part of Event-Based Workflows?

We implement a multi-layered approach to data protection.

‍1. Encryption: All data in transit is secured using TLS protocols, while data at rest is encrypted using industry-standard methods. This ensures that the information remains protected from unauthorised access.‍

2. Data Minimisation: We adhere to data minimisation principles, collecting and storing only the information necessary for threat detection. This reduces the potential exposure of sensitive data in case of a breach.‍

3. Auditing and Monitoring: Comprehensive logging and monitoring systems track access and actions taken within the platform. This creates a transparent audit trail, ensuring compliance and aiding in identifying potential vulnerabilities.‍

4. Regular Security Reviews: Our team conducts periodic security audits, penetration tests, and risk assessments to identify and mitigate any emerging threats or vulnerabilities in our systems.

These measures ensure the data involved in Event-Based Workflows is handled securely, aligning with the highest standards of data protection and compliance.

Future Developments and Innovations

What’s next for Event-Based Workflows? Are there any enhancements or additional capabilities you’re particularly excited about?

The next step for Event-Based Workflows is moving toward greater automation and intelligence. Currently, workflows need to be manually configured, which provides flexibility but requires time and expertise. In the future, we aim to leverage machine learning and advanced analytics to automatically set up and fine-tune workflows based on an organisation’s unique risk profile.

This enhancement would allow the platform to detect and respond to anomalous behaviour dynamically, without requiring pre-defined conditions. For example, it could automatically recognise unusual patterns like unexpected file sharing or data downloads, even if those behaviours don’t match existing rules. By tailoring workflows to each organisation’s needs and risk tolerance, we can make the platform not only more effective but also more accessible, helping organisations proactively address threats with minimal configuration effort.

Closing Thoughts

For security teams that are hesitant about implementing automated threat responses, what advice or reassurance can you offer?

It’s important to understand that automation is designed to augment, not replace, human oversight. Our Event-Based Workflows offer a range of configurable responses, from passive notifications to active interventions like user suspension. Teams can start small, using alerts to monitor incidents without taking immediate action, and scale up to more automated responses as they gain confidence in the system.

Additionally, automated responses are fully customisable, ensuring they align with your organisation’s risk tolerance and operational needs. For high-risk scenarios - like potential data exfiltration - automation acts as a safety net, providing quick, targeted responses that prevent harm while the security team investigates further. By starting with conservative settings and gradually increasing automation, teams can find a balance that enhances security without disrupting user workflows unnecessarily.

‍