SurePassID SIEM Connector (EventSyncLog) Architecture

SurePassID Authentication Server

SurePassID SIEM Connector (EventLogSync) Architecture

Overview

The SIEM Connector (EventLogSync) service provides a modular, extensible architecture for synchronizing event logs from SurePassID authentication servers to various SIEM (Security Information and Event Management) systems and log destinations.

This document describes the new service-based architecture introduced in version 2025.4, which refactors the monolithic EventLogSyncClient into a clean, interface-based design with separation of concerns.

Architecture Diagram

+---------------------------------------------------------------------------+
|                            EventLogSyncService                            |
|                           (Orchestration Layer)                           |
+---------------------------------------------------------------------------+
|                                                                           |
| +-------------------------+                 +---------------------------+ |
| |   IEventLogDataSource   |                 | IEventLogOutputFormatter  | |
| |         (Input)         |-EventLogRecord->|         (Output)          | |
| +-------------------------+                 +---------------------------+ |
|              |                                            |               |
|              v                                            v               |
| +-------------------------+                 +---------------------------+ |
| |  RestApiDataSource      |                 |  JsonFileOutputFormatter  | |
| |  (DatabaseDataSource)   |                 |  TextFileOutputFormatter  | |
| |  (WcfDataSource)        |                 |  WindowsEventLogFormatter | |
| +-------------------------+                 |  Log4NetOutputFormatter   | |
|                                             |  SyslogOutputFormatter    | |
|                                             +---------------------------+ |
|                                                                           |
+---------------------------------------------------------------------------+
                                      |
                                      v
                     +---------------------------------+
                     |       EventLogSyncFactory       |
                     |   (Component Creation Helper)   |
                     +---------------------------------+

Core Interfaces

1. IEventLogDataSource

Abstracts the source of event log data. Implementations fetch events from different backends.

Location: Client\EventLogSyncClientLib\Services\Interfaces\IEventLogDataSource.cs

public interface IEventLogDataSource : IDisposable
{
    string SourceName { get; }
    bool Initialize(IDictionary<string, string> config);
    FetchResult FetchNext(FetchOptions options);
    Task<FetchResult> FetchNextAsync(FetchOptions options, CancellationToken cancellationToken);
    bool MarkAsSynced(IEnumerable<EventLogRecord> records);
    Task<bool> MarkAsSyncedAsync(IEnumerable<EventLogRecord> records, CancellationToken cancellationToken);
    string LastError { get; }
    int LastErrorCode { get; }
}

Implementations: | Implementation | Description | Status | |---------------|-------------|--------| | RestApiDataSource | Fetches from SurePassID REST API | [OK] Implemented | | DatabaseDataSource | Direct SQL Server access | [Planned] | | WcfDataSource | Legacy WCF service | [Planned] |

2. IEventLogOutputFormatter

Abstracts the destination for event log data. Implementations write to different targets.

Location: Client\EventLogSyncClientLib\Services\Interfaces\IEventLogOutputFormatter.cs

public interface IEventLogOutputFormatter : IDisposable
{
    string FormatterName { get; }
    bool Initialize(IDictionary<string, string> config);
    WriteResult Write(EventLogRecord record, WriteOptions options);
    WriteResult WriteBatch(IEnumerable<EventLogRecord> records, WriteOptions options);
    Task<WriteResult> WriteAsync(EventLogRecord record, WriteOptions options, CancellationToken cancellationToken);
    Task<WriteResult> WriteBatchAsync(IEnumerable<EventLogRecord> records, WriteOptions options, CancellationToken cancellationToken);
    bool Flush();
    Task<bool> FlushAsync(CancellationToken cancellationToken);
    string LastError { get; }
}

Implementations: | Implementation | Description | Status | |---------------|-------------|--------| | JsonFileOutputFormatter | Structured JSON or NDJSON (Splunk) | [OK] Implemented | | TextFileOutputFormatter | Pipe-delimited text files | [OK] Implemented | | WindowsEventLogOutputFormatter | Windows Event Log | [OK] Implemented | | Log4NetOutputFormatter | Log4Net appenders (Syslog, etc.) | [Planned] | | SyslogOutputFormatter | Direct Syslog (RFC 5424) | [Planned] |

3. IEventLogSyncService

Orchestrates the synchronization process between data sources and output formatters.

Location: Client\EventLogSyncClientLib\Services\Interfaces\IEventLogSyncService.cs

public interface IEventLogSyncService : IDisposable
{
    IEventLogDataSource DataSource { get; set; }
    IEventLogOutputFormatter OutputFormatter { get; set; }
    SyncResult Synchronize(SyncOptions options);
    Task<SyncResult> SynchronizeAsync(SyncOptions options, CancellationToken cancellationToken);
    Task<SyncResult> SynchronizeAsync(SyncOptions options, IProgress<SyncProgress> progress, CancellationToken cancellationToken);
    void Stop();
    SyncProgress GetProgress();
    string LastError { get; }
    int LastErrorCode { get; }
    event EventHandler<SyncProgress> ProgressChanged;
    event EventHandler<SyncResult> SyncCompleted;
}

Data Transfer Objects

EventLogRecord

Common DTO representing a single event log entry, used across all data sources and output formatters.

public class EventLogRecord
{
    public string EventTime { get; set; }        // UTC timestamp (yyyy-MM-dd HH:mm:ss.fffZ)
    public string Tenant { get; set; }           // Tenant/account name
    public string Username { get; set; }         // User associated with event
    public string Action { get; set; }           // Action performed
    public string Severity { get; set; }         // Severity string (INFO, WARN, ERROR)
    public int SeverityCode { get; set; }        // Numeric severity
    public string AuthenticationMethod { get; set; }  // Auth method used
    public int ResultCode { get; set; }          // Numeric result code
    public string Description { get; set; }      // Event description
    public string IpV4 { get; set; }             // Client IPv4
    public string IpV6 { get; set; }             // Client IPv6
    public string SerialNumber { get; set; }     // Token serial number
    public string ApiKeyName { get; set; }       // API key name
    public string SsoIdentity { get; set; }      // SSO identity
    public string UserEmail { get; set; }        // User email
    public int? RecordId { get; set; }           // Database record ID
}

FetchResult

Result from data source fetch operations.

public class FetchResult
{
    public bool Success { get; set; }
    public int ErrorCode { get; set; }
    public string ErrorMessage { get; set; }
    public IReadOnlyList<EventLogRecord> Records { get; set; }
    public bool HasMoreRecords { get; set; }
    public int? TotalCount { get; set; }
}

WriteResult

Result from output formatter write operations.

public class WriteResult
{
    public bool Success { get; set; }
    public int RecordsWritten { get; set; }
    public string ErrorMessage { get; set; }
}

SyncResult

Result from sync service operations.

public class SyncResult
{
    public bool Success { get; set; }
    public int RecordsProcessed { get; set; }
    public int RecordsWritten { get; set; }
    public int RecordsFailed { get; set; }
    public string ErrorMessage { get; set; }
    public int ErrorCode { get; set; }
    public TimeSpan Duration { get; set; }
}

Component Details

RestApiDataSource

Fetches event logs from the SurePassID REST API.

Configuration Keys: | Key | Description | Required | |-----|-------------|----------| | apikeyid | API Key ID for authentication | Yes | | apikey | API Key secret | Yes | | restendpoint | REST API endpoint URL | Yes | | rest_api_trace | Enable request/response tracing | No |

Features:

• Supports both JSON and pipe-delimited response formats
• Automatic parsing of legacy pipe-delimited format
• Marks records as synced via API
• Handles pagination and rate limiting

JsonFileOutputFormatter

Writes event logs to JSON files.

Configuration Keys: | Key | Description | Default | |-----|-------------|---------| | exportdirectory | Output directory path | ./Export | | exportsourcename | Filename prefix | SEIM_Export | | outputformat | json or splunk | json |

Output Formats:

1. Structured JSON (outputformat=json)

{
  "runDate": "2025-01-15 10:30:00.000Z",
  "eventCount": 3,
  "syncEvents": [
    { "eventTime": "...", "tenant": "...", ... },
    { "eventTime": "...", "tenant": "...", ... }
  ]
}

2. NDJSON/Splunk (outputformat=splunk)

{"eventTime":"...","tenant":"...","username":"...",...}
{"eventTime":"...","tenant":"...","username":"...",...}

TextFileOutputFormatter

Writes event logs to pipe-delimited text files.

Configuration Keys: | Key | Description | Default | |-----|-------------|---------| | exportdirectory | Output directory path | ./Export | | exportsourcename | Filename prefix | SEIM_Export |

Output Format:

eventDate|tenantName|userName|auditAction|authMethod|severity|resultCode|ipV4|ipV6|serialNumber|apiKeyName|result|ssoIdentity|userEmail

WindowsEventLogOutputFormatter

Writes event logs to Windows Event Log.

Configuration Keys: | Key | Description | Default | |-----|-------------|---------| | eventsourcename | Event source name | SEIM_Export | | eventlogname | Event log name | SurePass | | outputformat | text or json | text |

Notes:

• Requires administrator privileges to create event source
• Maps severity codes to Windows Event Log entry types

EventLogSyncService

Main orchestration service that coordinates data sources and output formatters.

Features:

• Async/await support with cancellation tokens
• Progress reporting via events and IProgress
• Batch processing with configurable batch sizes
• Continuous sync mode with configurable wait intervals
• Automatic retry on transient failures

Factory Pattern

The EventLogSyncFactory provides helper methods for creating configured service components.

// Create from configuration dictionary
var dataSource = EventLogSyncFactory.CreateDataSource(DataSourceType.RestApi, config);
var formatter = EventLogSyncFactory.CreateOutputFormatter(OutputFormatterType.JsonFile, config);
var service = new EventLogSyncService(dataSource, formatter);

// Create from legacy parameters
var service = EventLogSyncFactory.CreateFromLegacyParameters(stringDictionary);
var options = EventLogSyncFactory.CreateSyncOptions(config);

Data Flow

+-------------------+       +-------------------+       +-------------------+
|     REST API      |       |   EventLogSync    |       |     JSON File     |
|      Server       |------>|      Service      |------>|    SIEM System    |
|                   |       |                   |       |     Event Log     |
+-------------------+       +-------------------+       +-------------------+
          |                           |                           |
          v                           v                           v
   +-------------+             +-------------+             +-------------+
   | FetchResult |             |  EventLog   |             | WriteResult |
   |  .Records   |             |  Record[]   |             |  .Success   |
   +-------------+             +-------------+             +-------------+

1. Fetch Phase: DataSource fetches records from the server
2. Transform Phase: Records are converted to common EventLogRecord format
3. Write Phase: OutputFormatter writes records to destination
4. Mark Phase: DataSource marks records as synced

REST API Formats

The REST API supports two response formats:

Pipe-Delimited (Default)

2025-01-15 10:30:00.000Z|TenantName|user@example.com|OTP Validation|OTP|INFO|0|192.168.1.1||SN123456|ApiKey1|Success

JSON Format (restapiformat=json)

{
  "eventTime": "2025-01-15 10:30:00.000Z",
  "tenant": "TenantName",
  "username": "user@example.com",
  "action": "OTP Validation",
  "severity": "INFO",
  "eventResultCode": 0,
  "eventDescription": "Success",
  "eventIpV4": "192.168.1.1",
  "eventAuthenticationMethod": "OTP",
  "eventSerialNumber": "SN123456",
  "eventApiKeyName": "ApiKey1"
}

Configuration

Full Configuration Example

<appSettings>
  <!-- Authentication -->
  <add key="Server.ApiKeyId" value="your-api-key-id" />
  <add key="Server.ApiKey" value="your-api-key-secret" />
  
  <!-- Data Source -->
  <add key="Server.SyncApi" value="REST" />
  <add key="Server.RESTEndPoint" value="https://api.surepassid.com/api/mfa/v1" />
  <add key="Server.RestApiFormat" value="json" />
  
  <!-- Output -->
  <add key="Server.SyncTargetType" value="filesystem" />
  <add key="Server.SyncTargetFileSystemFormat" value="json" />
  
  <!-- Sync Options -->
  <add key="Server.MaxSyncItems" value="1000" />
  <add key="Server.RunOption" value="continuous" />
  <add key="Server.ContinuousRunWaitTimeSeconds" value="30" />
  <add key="Server.IgnoreSyncStatus" value="false" />
  
  <!-- Date Filters (optional) -->
  <add key="Server.StartDateUtc" value="2025-01-01" />
  <add key="Server.EndDateUtc" value="2025-12-31" />
</appSettings>

Command-Line Parameters

EventLogSyncClientConsole.exe \
  -apikeyid=your-api-key-id \
  -apikey=your-api-key-secret \
  -syncapi=REST \
  -restendpoint=https://api.surepassid.com/api/mfa/v1 \
  -targettype=filesystem \
  -targetfilesystemformat=json \
  -maxsyncitems=1000 \
  -runoption=once \
  -restapiformat=json

Error Handling

Error Codes

Retry Strategy

The service implements exponential backoff for transient failures:

• Initial delay: 1 second
• Maximum delay: 30 seconds
• Maximum retries: 3

Extensibility

Adding a New Data Source

1. Create a class implementing IEventLogDataSource
2. Implement the required methods
3. Add to DataSourceType enum
4. Update EventLogSyncFactory.CreateDataSource()

public class CustomDataSource : IEventLogDataSource
{
    public string SourceName => "Custom Source";
    
    public bool Initialize(IDictionary<string, string> config)
    {
        // Initialize from config
        return true;
    }
    
    public FetchResult FetchNext(FetchOptions options)
    {
        // Fetch records from custom source
        return FetchResult.Succeeded(records);
    }
    
    // ... implement other methods
}

Adding a New Output Formatter

1. Create a class implementing IEventLogOutputFormatter
2. Implement the required methods
3. Add to OutputFormatterType enum
4. Update EventLogSyncFactory.CreateOutputFormatter()

public class CustomOutputFormatter : IEventLogOutputFormatter
{
    public string FormatterName => "Custom Formatter";
    
    public bool Initialize(IDictionary<string, string> config)
    {
        // Initialize from config
        return true;
    }
    
    public WriteResult WriteBatch(IEnumerable<EventLogRecord> records, WriteOptions options)
    {
        // Write records to custom destination
        return WriteResult.Succeeded(count);
    }
    
    // ... implement other methods
}

Testing

Test Categories

Running Tests

# Run all tests
dotnet test EventLogSyncTests.csproj

# Run specific category
dotnet test --filter "TestCategory=Services"

# Run integration tests
dotnet test --filter "TestCategory=Integration"

Migration from Legacy Architecture

The new service-based architecture is backward compatible with the legacy EventLogSyncClient class.

Legacy Code

var options = new EventLogSyncClientOptions();
options.Init(parameters);
EventLogSyncClient.Synchronize(options);

New Service Architecture

var service = EventLogSyncFactory.CreateFromLegacyParameters(parameters);
var options = EventLogSyncFactory.CreateSyncOptions(configDict);
var result = await service.SynchronizeAsync(options);

File Structure

Client\EventLogSyncClientLib\
+-- Services\
|   +-- Interfaces\
|   |   +-- IEventLogDataSource.cs       # Data source interface
|   |   +-- IEventLogOutputFormatter.cs  # Output formatter interface
|   |   \-- IEventLogSyncService.cs      # Sync service interface
|   +-- DataSources\
|   |   \-- RestApiDataSource.cs         # REST API implementation
|   +-- OutputFormatters\
|   |   +-- JsonFileOutputFormatter.cs   # JSON file output
|   |   +-- TextFileOutputFormatter.cs   # Text file output
|   |   \-- WindowsEventLogOutputFormatter.cs  # Windows Event Log
|   +-- EventLogSyncService.cs           # Main sync service
|   \-- EventLogSyncFactory.cs           # Factory for creating components
+-- EventLogClient.cs                    # Legacy monolithic client (preserved)
+-- EventLogSyncJsonObjects.cs           # JSON serialization DTOs
\-- Util.cs                              # Utility functions

Version History