In Logstash configurations, field names prefixed with @ represent core metadata fields that have special significance in the processing pipeline. These fields are typically used for:

@timestamp - Event timestamp (automatically added by Logstash)
@version  - Event format version
@source_host - Originating host information
@message  - The raw log message content
@fields   - Custom fields namespace

The @ prefix serves multiple technical purposes:

• Identifies system-managed metadata fields
• Prevents namespace collisions with application data
• Ensures consistent field naming across different inputs
• Provides standard fields expected by outputs (especially Elasticsearch)

The sample configuration demonstrates several key uses:

# Source host normalization
mutate {
  rename => [ "Hostname", "@source_host" ]
}

# Message field standardization
mutate {
  rename => [ "Message", "@message" ]
}

# Timestamp processing (implicit @timestamp field)
date {
  EventReceivedTime => "UNIX"
}

When sending data to Elasticsearch, these special fields map to:

When working with Logstash fields:

1. Use @ prefix only for core metadata fields
2. For custom fields, use descriptive names without special prefixes
3. Maintain consistency across your pipeline configurations
4. Document your field naming conventions

Developers often encounter these issues:

# Problem: Overwriting system fields
mutate {
  add_field => { "@timestamp" => "manual value" } # BAD
}

# Solution: Use custom fields instead
mutate {
  add_field => { "processed_timestamp" => "%{+YYYY.MM.dd}" } # GOOD
}

For complex scenarios, you might:

# Conditionally modify @message
filter {
  if [type] == "syslog" {
    mutate {
      replace => [ "@message", "SYSLOG: %{message}" ]
    }
  }
}

# Copy between fields
mutate {
  rename => { "original_host" => "@source_host" }
  copy   => { "@source_host" => "backup_host" }
}

In Logstash configurations, fields prefixed with @ hold special significance as they represent core metadata fields within the event processing pipeline. These fields form the fundamental structure of Logstash events:

{
  "@timestamp": "2023-01-01T12:00:00.000Z",
  "@version": "1",
  "@metadata": {...},
  "regular_field": "value"
}

The most commonly encountered special fields include:

• @timestamp: The exact time when the event occurred
• @version: Event format version
• @metadata: Contains non-indexed processing data
• @source_host: Origin host of the event
• @message: The raw message content

The configuration example shows intentional renaming to these special fields for important processing reasons:

mutate {
  rename => [ "Hostname", "@source_host" ]
  rename => [ "Message", "@message" ]
}

This achieves:

• Standardization: Matches Logstash's expected field naming conventions
• Downstream Compatibility: Ensures proper handling by outputs like Elasticsearch
• Metadata Preservation: Maintains critical processing information

Here's how these fields are typically transformed through the pipeline:

filter {
  mutate {
    add_field => { 
      "received_at" => "%{@timestamp}"
      "source_ip" => "%{@source_host}"
    }
  }
  
  date {
    match => [ "log_timestamp", "ISO8601" ]
    target => "@timestamp"
  }
}

Field Reference Syntax: Note the different reference patterns:

# Accessing special fields:
filter {
  mutate { add_field => { "debug" => "%{@timestamp}" } }
}

# Regular field access:
filter {
  mutate { lowercase => [ "normal_field" ] }
}

Output Handling: Many outputs treat @-prefixed fields specially:

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "logs-%{+YYYY.MM.dd}"
    document_type => "%{@type}"
  }
}

Common problems and solutions:

# Problem: @timestamp parsing failure
filter {
  date {
    match => [ "event_time", "UNIX" ]
    target => "@timestamp"
  }
}

# Problem: Missing @source_host
filter {
  mutate {
    rename => { "original_host" => "@source_host" }
    add_field => { "@source_host" => "unknown" }
  }
}