Forum Discussion

venkat's avatar
venkat
Icon for Neophyte rankNeophyte
24 days ago

OCI APplication log Monitoring

Hi All,

 

We are trying to Monitor OCI Application log Monitoring in LM,we have setup the fluentd config and Updated the COnfig file as well,but still we dont see the logs captured in LM.The Log file has all the reqired permissions.
Here is the output of Fluentd.log file

2024-10-28 03:08:01 -0500 [info]: #0 flushing all buffer forcedly
2024-10-28 11:40:21 -0500 [info]: Received graceful stop
2024-10-28 11:40:21 -0500 [info]: Received graceful stop
2024-10-28 11:40:21 -0500 [info]: #0 fluentd worker is now stopping worker=0
2024-10-28 11:40:21 -0500 [info]: #0 shutting down fluentd worker worker=0
2024-10-28 11:40:21 -0500 [info]: #0 shutting down input plugin type=:tail plugin_id="object:8c0"
2024-10-28 11:40:21 -0500 [info]: #0 shutting down output plugin type=:lm plugin_id="object:898"
2024-10-28 11:40:21 -0500 [info]: #0 shutting down filter plugin type=:record_transformer plugin_id="object:870"
2024-10-28 11:40:21 -0500 [info]: Worker 0 finished with status 0
2024-10-28 11:40:22 -0500 [info]: init supervisor logger path="/var/log/fluent/fluentd.log" rotate_age=nil rotate_size=nil
2024-10-28 11:40:22 -0500 [info]: parsing config file is succeeded path="/etc/fluent/fluentd.conf"
2024-10-28 11:40:22 -0500 [info]: gem 'fluentd' version '1.16.5'
2024-10-28 11:40:22 -0500 [info]: gem 'fluent-plugin-calyptia-monitoring' version '0.1.3'
2024-10-28 11:40:22 -0500 [info]: gem 'fluent-plugin-elasticsearch' version '5.4.0'
2024-10-28 11:40:22 -0500 [info]: gem 'fluent-plugin-flowcounter-simple' version '0.1.0'
2024-10-28 11:40:22 -0500 [info]: gem 'fluent-plugin-kafka' version '0.19.2'
2024-10-28 11:40:22 -0500 [info]: gem 'fluent-plugin-lm-logs' version '1.2.2'
2024-10-28 11:40:22 -0500 [info]: gem 'fluent-plugin-metrics-cmetrics' version '0.1.2'
2024-10-28 11:40:22 -0500 [info]: gem 'fluent-plugin-opensearch' version '1.1.4'
2024-10-28 11:40:22 -0500 [info]: gem 'fluent-plugin-prometheus' version '2.1.0'
2024-10-28 11:40:22 -0500 [info]: gem 'fluent-plugin-prometheus_pushgateway' version '0.1.1'
2024-10-28 11:40:22 -0500 [info]: gem 'fluent-plugin-record-modifier' version '2.1.1'
2024-10-28 11:40:22 -0500 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.4.0'
2024-10-28 11:40:22 -0500 [info]: gem 'fluent-plugin-s3' version '1.7.2'
2024-10-28 11:40:22 -0500 [info]: gem 'fluent-plugin-sd-dns' version '0.1.0'
2024-10-28 11:40:22 -0500 [info]: gem 'fluent-plugin-systemd' version '1.0.5'
2024-10-28 11:40:22 -0500 [info]: gem 'fluent-plugin-td' version '1.2.0'
2024-10-28 11:40:22 -0500 [info]: gem 'fluent-plugin-utmpx' version '0.5.0'
2024-10-28 11:40:22 -0500 [info]: gem 'fluent-plugin-webhdfs' version '1.5.0'
2024-10-28 11:40:22 -0500 [info]: using configuration file: <ROOT>
  <source>
    @type tail
    path "/xxxx/xxxxx/xxx/xx/xxx/xxx/xxx/xxxx/xxx/xxx/xxxxx/xxxx/server1.out"
    pos_file "/tmp/server1.pos"
    tag "xxxxx_log"
    <parse>
      @type "multiline"
      format_firstline "/^(?<time>\\w{3} \\d{2}, \\d{4} \\d{2}:\\d{2}:\\d{2} [APM]{2})$/"
      format1 /^(?<message>.*java.lang.OutOfMemoryError*)$/
      unmatched_lines 
    </parse>
  </source>
  <filter OCI.**>
    @type record_transformer
    <record>
      hostname xxxxxx
      tag ${tag}
    </record>
  </filter>
  <match OCI.**>
    @type lm
    resource_mapping {"hostname":"system.displayName"}
    company_name "xxx"
    access_id "xxxxxxxxxxxx"
    access_key xxxxxx
    debug true
    compression "gzip"
    include_metadata true
    <buffer>
      @type "memory"
      flush_interval 1s
      chunk_limit_size 5m
    </buffer>
  </match>
</ROOT>
2024-10-28 11:40:22 -0500 [info]: starting fluentd-1.16.5 pid=3684 ruby="3.2.3"
2024-10-28 11:40:22 -0500 [info]: spawn command to main:  cmdline=["/opt/fluent/bin/ruby", "-Eascii-8bit:ascii-8bit", "/opt/fluent/bin/fluentd", "--log", "/var/log/fluent/fluentd.log", "--daemon", "/var/run/fluent/fluentd.pid", "--under-supervisor"]
2024-10-28 11:40:23 -0500 [info]: #0 init worker0 logger path="/var/log/fluent/fluentd.log" rotate_age=nil rotate_size=nil
2024-10-28 11:40:23 -0500 [info]: adding filter pattern="OCI.**" type="record_transformer"
2024-10-28 11:40:23 -0500 [info]: adding match pattern="OCI.**" type="lm"
2024-10-28 11:40:23 -0500 [info]: adding source type="tail"
2024-10-28 11:40:23 -0500 [info]: #0 starting fluentd worker pid=3761 ppid=3756 worker=0
2024-10-28 11:40:23 -0500 [info]: #0 fluentd worker is now running worker=0

No RepliesBe the first to reply