#

Note

VictoriaTraces is currently under active development and not ready for production use. It is built on top of VictoriaLogs and therefore shares some flags and APIs. These will be fully separated once VictoriaTraces reaches a stable release. Until then, features may change or break without notice.

VictoriaTraces can be queried with LogsQL via the following ways:

Web UI - a web-based UI for querying trace spans
HTTP API

And on top of the LogsQL, VictoriaTraces also implemented Jaeger Query Service JSON APIs . These APIs can be used with:

Web UI #

VictoriaTraces provides Web UI for trace spans querying and exploration at http://<victoria-traces>:9428/select/vmui.

There are three modes of displaying query results:

Group - results are displayed as a table with rows grouped by stream fields .
Table - displays query results as a table.
JSON - displays raw JSON response from /select/logsql/query HTTP API.

HTTP API #

VictoriaTraces provides the same HTTP endpoints that VictoriaLogs provides. Additionally, the following Jaeger HTTP endpoints are available:

/select/jaeger/api/services for querying all the services
/select/jaeger/api/services/{service_name}/operations for querying all the span names of a service.
/select/jaeger/api/traces for querying traces.
/select/jaeger/api/traces/{trace_id} for querying a trace.
/select/jaeger/api/dependencies for querying dependencies, currently under developing.

Querying traces #

Trace spans in VictoriaTraces can be queried at the The /select/jaeger/api/traces HTTP endpoint. This endpoint provides the following params:

service: the service name.
operation: the span name (also known as the operation name in Jaeger).
tags: the attributes (also known as tags) filter, example: {"key":"value"}
start: the start timestamp in unix microseconds.
end: the end timestamp in unix microseconds.
minDuration: the minimum duration of the span, with units ns, us, ms, s, m, or h.
maxDuration: the maximum duration of the span, with units ns, us, ms, s, m, or h.
limit: the trace limit of the query, default 20.

For example, the following queries are typically how users try to find a specific trace:

List all the services:

      curl http://<victoria-traces>:9428/select/jaeger/api/services

Here’s a response example:

      {
    "data": [
        "accounting",
        "ad",
        "cart",
        "checkout",
        "currency"
    ],
    "errors": null,
    "limit": 0,
    "offset": 0,
    "total": 5
}
    

List all the span names of a service:

      curl http://<victoria-traces>:9428/select/jaeger/api/services/checkout/operations

Here’s a response example:

      {
    "data": [
        "HTTP POST",
        "orders publish",
        "oteldemo.CartService/EmptyCart",
        "oteldemo.CartService/GetCart",
        "oteldemo.CheckoutService/PlaceOrder",
    ],
    "errors": null,
    "limit": 0,
    "offset": 0,
    "total": 5
}
    

Filter traces by conditions:

      curl http://<victoria-traces>:9428/select/jaeger/api/traces?service=checkout&operation=oteldemo&tags=%7B%22rpc.method%22%3A%22Convert%22%7D&minDuration=1ms&maxDuration=10ms&limit=5&start=1749969952453000&end=1750056352453000
    

Here’s a response example:

      {"data":[{"processes":{"p1":{"serviceName":"email","tags":[{"key":"process.command","type":"string","value":"email_server.rb"},{"key":"process.pid","type":"string","value":"1"},{"key":"process.runtime.description","type":"string","value":"ruby 3.4.4 (2025-05-14 revision a38531fd3f) +PRISM [aarch64-linux-musl]"},{"key":"process.runtime.name","type":"string","value":"ruby"},{"key":"process.runtime.version","type":"string","value":"3.4.4"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"ruby"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.8.0"}]},"p10":{"serviceName":"cart","tags":[{"key":"container.id","type":"string","value":"5603ff989877ecf311403b6ea81fda10734846a0cbdad3a09c39fb068e4a07fc"},{"key":"host.name","type":"string","value":"5603ff989877"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"dotnet"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.11.2"}]},"p11":{"serviceName":"product-catalog","tags":[{"key":"host.name","type":"string","value":"3dabfcfe8381"},{"key":"os.description","type":"string","value":"Debian GNU/Linux Debian GNU/Linux 12 (bookworm) (Linux 3dabfcfe8381 6.10.14-linuxkit #1 SMP Tue Apr 15 16:00:54 UTC 2025 aarch64)"},{"key":"os.type","type":"string","value":"linux"},{"key":"process.command_args","type":"string","value":"[\"./product-catalog\"]"},{"key":"process.executable.name","type":"string","value":"product-catalog"},{"key":"process.executable.path","type":"string","value":"/usr/src/app/product-catalog"},{"key":"process.owner","type":"string","value":"nonroot"},{"key":"process.pid","type":"string","value":"1"},{"key":"process.runtime.description","type":"string","value":"go version go1.24.4 linux/arm64"},{"key":"process.runtime.name","type":"string","value":"go"},{"key":"process.runtime.version","type":"string","value":"go1.24.4"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"go"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.36.0"}]},"p12":{"serviceName":"currency","tags":[{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"cpp"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.20.0"}]},"p2":{"serviceName":"quote","tags":[{"key":"container.id","type":"string","value":"759183873eeb1328f16df8ea5b5a10932506af136a6537c6a365131c04f1645c"},{"key":"host.arch","type":"string","value":"aarch64"},{"key":"host.name","type":"string","value":"759183873eeb"},{"key":"os.description","type":"string","value":"6.10.14-linuxkit"},{"key":"os.name","type":"string","value":"Linux"},{"key":"os.type","type":"string","value":"linux"},{"key":"os.version","type":"string","value":"#1 SMP Tue Apr 15 16:00:54 UTC 2025"},{"key":"process.command","type":"string","value":"public/index.php"},{"key":"process.command_args","type":"string","value":"[\"public/index.php\"]"},{"key":"process.executable.path","type":"string","value":"/usr/local/bin/php"},{"key":"process.owner","type":"string","value":"www-data"},{"key":"process.pid","type":"string","value":"1"},{"key":"process.runtime.name","type":"string","value":"cli"},{"key":"process.runtime.version","type":"string","value":"8.3.22"},{"key":"service.instance.id","type":"string","value":"9dc0abaa-c408-483e-9fed-8375a73efb91"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.distro.name","type":"string","value":"opentelemetry-php-instrumentation"},{"key":"telemetry.distro.version","type":"string","value":"1.1.3"},{"key":"telemetry.sdk.language","type":"string","value":"php"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.5.0"}]},"p3":{"serviceName":"checkout","tags":[{"key":"host.name","type":"string","value":"cbdb5e0808c2"},{"key":"os.description","type":"string","value":"Debian GNU/Linux Debian GNU/Linux 12 (bookworm) (Linux cbdb5e0808c2 6.10.14-linuxkit #1 SMP Tue Apr 15 16:00:54 UTC 2025 aarch64)"},{"key":"os.type","type":"string","value":"linux"},{"key":"process.command_args","type":"string","value":"[\"./checkout\"]"},{"key":"process.executable.name","type":"string","value":"checkout"},{"key":"process.executable.path","type":"string","value":"/usr/src/app/checkout"},{"key":"process.owner","type":"string","value":"nonroot"},{"key":"process.pid","type":"string","value":"1"},{"key":"process.runtime.description","type":"string","value":"go version go1.24.4 linux/arm64"},{"key":"process.runtime.name","type":"string","value":"go"},{"key":"process.runtime.version","type":"string","value":"go1.24.4"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"go"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.36.0"}]},"p4":{"serviceName":"frontend","tags":[{"key":"container.id","type":"string","value":"2d395f01353040612a00252cf6e8c32f00ab94ae06f82f143a3ea9c742072674"},{"key":"host.arch","type":"string","value":"arm64"},{"key":"host.name","type":"string","value":"2d395f013530"},{"key":"os.type","type":"string","value":"linux"},{"key":"os.version","type":"string","value":"6.10.14-linuxkit"},{"key":"process.command","type":"string","value":"/app/server.js"},{"key":"process.command_args","type":"string","value":"[\"/usr/local/bin/node\",\"--require\",\"./Instrumentation.js\",\"/app/server.js\"]"},{"key":"process.executable.name","type":"string","value":"node"},{"key":"process.executable.path","type":"string","value":"/usr/local/bin/node"},{"key":"process.owner","type":"string","value":"nextjs"},{"key":"process.pid","type":"string","value":"17"},{"key":"process.runtime.description","type":"string","value":"Node.js"},{"key":"process.runtime.name","type":"string","value":"nodejs"},{"key":"process.runtime.version","type":"string","value":"22.16.0"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"nodejs"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.30.1"}]},"p5":{"serviceName":"shipping","tags":[{"key":"os.type","type":"string","value":"linux"},{"key":"process.command_args","type":"string","value":"[\"/app/shipping\"]"},{"key":"process.pid","type":"string","value":"1"},{"key":"process.runtime.description","type":"string","value":"rustc 1.82.0 (f6e511eec 2024-10-15)"},{"key":"process.runtime.name","type":"string","value":"rustc"},{"key":"process.runtime.version","type":"string","value":"1.82.0"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"rust"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"0.30.0"}]},"p6":{"serviceName":"payment","tags":[{"key":"container.id","type":"string","value":"18ee03279d38ed0e0eedad037c260df78dfc3323aa662ca14a2d38fcc8bf3762"},{"key":"host.arch","type":"string","value":"arm64"},{"key":"host.name","type":"string","value":"18ee03279d38"},{"key":"os.type","type":"string","value":"linux"},{"key":"os.version","type":"string","value":"6.10.14-linuxkit"},{"key":"process.command","type":"string","value":"/usr/src/app/index.js"},{"key":"process.command_args","type":"string","value":"[\"/usr/local/bin/node\",\"--require\",\"./opentelemetry.js\",\"/usr/src/app/index.js\"]"},{"key":"process.executable.name","type":"string","value":"node"},{"key":"process.executable.path","type":"string","value":"/usr/local/bin/node"},{"key":"process.owner","type":"string","value":"node"},{"key":"process.pid","type":"string","value":"17"},{"key":"process.runtime.description","type":"string","value":"Node.js"},{"key":"process.runtime.name","type":"string","value":"nodejs"},{"key":"process.runtime.version","type":"string","value":"22.16.0"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"nodejs"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.30.1"}]},"p7":{"serviceName":"flagd","tags":[{"key":"host.name","type":"string","value":"1f315d8a0f78"},{"key":"os.description","type":"string","value":"Debian GNU/Linux Debian GNU/Linux 12 (bookworm) (Linux 1f315d8a0f78 6.10.14-linuxkit #1 SMP Tue Apr 15 16:00:54 UTC 2025 aarch64)"},{"key":"os.type","type":"string","value":"linux"},{"key":"process.runtime.version","type":"string","value":"go1.24.1"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"v0.12.3"},{"key":"telemetry.sdk.language","type":"string","value":"go"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.35.0"}]},"p8":{"serviceName":"load-generator","tags":[{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"python"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.34.0"}]},"p9":{"serviceName":"frontend-proxy","tags":[{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"}]}},"spans":[{"duration":4935,"logs":[],"operationName":"send_email","processID":"p1","references":[{"refType":"CHILD_OF","spanID":"739cd04d718779ae","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"032bf7007e123e8d","startTime":1750044449769690,"tags":[{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"email"},{"key":"error","type":"string","value":"unset"},{"key":"app.email.recipient","type":"string","value":"reed@example.com"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":283,"logs":[],"operationName":"sinatra.render_template","processID":"p1","references":[{"refType":"CHILD_OF","spanID":"1fd5f529c2dd316b","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"bc5f262c2f7d9bb5","startTime":1750044449770317,"tags":[{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"OpenTelemetry::Instrumentation::Sinatra"},{"key":"otel.scope.version","type":"string","value":"0.25.0"},{"key":"error","type":"string","value":"unset"},{"key":"sinatra.template_name","type":"string","value":"layout"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":961,"logs":[],"operationName":"sinatra.render_template","processID":"p1","references":[{"refType":"CHILD_OF","spanID":"032bf7007e123e8d","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"1fd5f529c2dd316b","startTime":1750044449769761,"tags":[{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"OpenTelemetry::Instrumentation::Sinatra"},{"key":"otel.scope.version","type":"string","value":"0.25.0"},{"key":"error","type":"string","value":"unset"},{"key":"sinatra.template_name","type":"string","value":"confirmation"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":3339,"logs":[{"timestamp":1750044449717803,"fields":[{"key":"event","type":"string","value":"Received get quote request, processing it"}]},{"timestamp":1750044449718100,"fields":[{"key":"event","type":"string","value":"Quote processed, response sent back"},{"key":"app.quote.cost.total","type":"string","value":"227.5"}]}],"operationName":"{closure}","processID":"p2","references":[{"refType":"CHILD_OF","spanID":"aaf29afb62662d95","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"ea80042fbe6e5887","startTime":1750044449717692,"tags":[{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"io.opentelemetry.contrib.php.slim"},{"key":"code.file.path","type":"string","value":"/var/www/vendor/php-di/slim-bridge/src/ControllerInvoker.php"},{"key":"code.function.name","type":"string","value":"DI\\Bridge\\Slim\\ControllerInvoker::__invoke"},{"key":"code.line.number","type":"string","value":"29"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":6755,"logs":[],"operationName":"oteldemo.PaymentService/Charge","processID":"p3","references":[{"refType":"CHILD_OF","spanID":"7683762fa74ffd1c","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"530667cc212dd6ed","startTime":1750044449739280,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"Charge"},{"key":"rpc.service","type":"string","value":"oteldemo.PaymentService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"server.address","type":"string","value":"172.18.0.14"},{"key":"server.port","type":"string","value":"50051"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":6544,"logs":[],"operationName":"POST /getquote","processID":"p2","references":[{"refType":"CHILD_OF","spanID":"09b03b9b5481c29c","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"aaf29afb62662d95","startTime":1750044449717102,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"io.opentelemetry.contrib.php.slim"},{"key":"code.file.path","type":"string","value":"/var/www/vendor/slim/slim/Slim/App.php"},{"key":"code.function.name","type":"string","value":"Slim\\App::handle"},{"key":"code.line.number","type":"string","value":"207"},{"key":"http.request.body.size","type":"string","value":"19"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.body.size","type":"string","value":"-"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"http.route","type":"string","value":"/getquote"},{"key":"network.protocol.version","type":"string","value":"1.1"},{"key":"server.address","type":"string","value":"quote"},{"key":"server.port","type":"string","value":"8090"},{"key":"url.full","type":"string","value":"http://quote:8090/getquote"},{"key":"url.path","type":"string","value":"/getquote"},{"key":"url.scheme","type":"string","value":"http"},{"key":"user_agent.original","type":"string","value":"-"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":77220,"logs":[],"operationName":"executing api route (pages) /api/checkout","processID":"p4","references":[{"refType":"CHILD_OF","spanID":"01468af9419620f5","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"6b73da57ebca1b82","startTime":1750044449702000,"tags":[{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"next.js"},{"key":"otel.scope.version","type":"string","value":"0.0.1"},{"key":"http.status_code","type":"string","value":"200"},{"key":"next.span_name","type":"string","value":"executing api route (pages) /api/checkout"},{"key":"next.span_type","type":"string","value":"Node.runHandler"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":1831,"logs":[],"operationName":"oteldemo.CartService/GetCart","processID":"p3","references":[{"refType":"CHILD_OF","spanID":"96f2298052cc3fda","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"111cb151fdd9a915","startTime":1750044449708652,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"GetCart"},{"key":"rpc.service","type":"string","value":"oteldemo.CartService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"server.address","type":"string","value":"172.18.0.10"},{"key":"server.port","type":"string","value":"7070"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":46,"logs":[],"operationName":"/ship-order","processID":"p5","references":[{"refType":"CHILD_OF","spanID":"92345ad5d7cb4190","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"d1253691f90f5b95","startTime":1750044449746781,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"opentelemetry-instrumentation-actix-web"},{"key":"otel.scope.version","type":"string","value":"0.22.0"},{"key":"client.address","type":"string","value":"172.18.0.23"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"http.route","type":"string","value":"/ship-order"},{"key":"network.protocol.version","type":"string","value":"1.1"},{"key":"server.address","type":"string","value":"shipping"},{"key":"server.port","type":"string","value":"50050"},{"key":"url.path","type":"string","value":"/ship-order"},{"key":"url.scheme","type":"string","value":"http"},{"key":"user_agent.original","type":"string","value":"Go-http-client/1.1"},{"key":"error","type":"string","value":"unset"},{"key":"messaging.message.body.size","type":"string","value":"182"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":78153,"logs":[],"operationName":"POST","processID":"p4","references":[{"refType":"CHILD_OF","spanID":"df1b3d5c8e0ab6be","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"47c48aa63a0c5a3d","startTime":1750044449701000,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"@opentelemetry/instrumentation-http"},{"key":"otel.scope.version","type":"string","value":"0.57.1"},{"key":"http.flavor","type":"string","value":"1.1"},{"key":"http.host","type":"string","value":"frontend-proxy:8080"},{"key":"http.method","type":"string","value":"POST"},{"key":"http.scheme","type":"string","value":"http"},{"key":"http.status_code","type":"string","value":"200"},{"key":"http.user_agent","type":"string","value":"python-requests/2.32.4"},{"key":"net.host.name","type":"string","value":"frontend-proxy"},{"key":"net.peer.ip","type":"string","value":"172.18.0.26"},{"key":"net.transport","type":"string","value":"ip_tcp"},{"key":"error","type":"string","value":"unset"},{"key":"http.request_content_length_uncompressed","type":"string","value":"388"},{"key":"http.status_text","type":"string","value":"OK"},{"key":"http.target","type":"string","value":"/api/checkout"},{"key":"http.url","type":"string","value":"http://frontend-proxy:8080/api/checkout"},{"key":"net.host.ip","type":"string","value":"172.18.0.24"},{"key":"net.host.port","type":"string","value":"8080"},{"key":"net.peer.port","type":"string","value":"35632"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":1988,"logs":[],"operationName":"charge","processID":"p6","references":[{"refType":"CHILD_OF","spanID":"df89f1712cb9fdec","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"f30e92001c694787","startTime":1750044449743000,"tags":[{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"payment"},{"key":"app.payment.card_type","type":"string","value":"visa"},{"key":"app.payment.card_valid","type":"string","value":"true"},{"key":"app.payment.charged","type":"string","value":"false"},{"key":"error","type":"string","value":"unset"},{"key":"app.loyalty.level","type":"string","value":"silver"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":128,"logs":[{"timestamp":1750044449717887,"fields":[{"key":"event","type":"string","value":"Calculating quote"}]},{"timestamp":1750044449717919,"fields":[{"key":"event","type":"string","value":"Quote calculated, returning its value"}]}],"operationName":"calculate-quote","processID":"p2","references":[{"refType":"CHILD_OF","spanID":"ea80042fbe6e5887","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"0b119b964828c67b","startTime":1750044449717886,"tags":[{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"manual-instrumentation"},{"key":"error","type":"string","value":"unset"},{"key":"app.quote.cost.total","type":"string","value":"227.5"},{"key":"app.quote.items.count","type":"string","value":"5"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":6,"logs":[],"operationName":"resolveBoolean","processID":"p7","references":[{"refType":"CHILD_OF","spanID":"3af2ca071042ef47","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"ab8c870e76bbe57f","startTime":1750044449753032,"tags":[{"key":"error","type":"string","value":"unset"},{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"jsonEvaluator"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":70,"logs":[],"operationName":"resolveBoolean","processID":"p7","references":[{"refType":"CHILD_OF","spanID":"9d054ff4aeb2b518","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"3af2ca071042ef47","startTime":1750044449753027,"tags":[{"key":"error","type":"string","value":"unset"},{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"flagd.evaluation.v1"},{"key":"feature_flag.key","type":"string","value":"cartFailure"},{"key":"feature_flag.provider_name","type":"string","value":"flagd"},{"key":"feature_flag.variant","type":"string","value":"off"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":69871,"logs":[{"timestamp":1750044449737830,"fields":[{"key":"event","type":"string","value":"prepared"}]},{"timestamp":1750044449739261,"fields":[{"key":"feature_flag.key","type":"string","value":"paymentUnreachable"},{"key":"feature_flag.provider_name","type":"string","value":"flagd"},{"key":"feature_flag.variant","type":"string","value":"off"},{"key":"event","type":"string","value":"feature_flag"}]},{"timestamp":1750044449746517,"fields":[{"key":"event","type":"string","value":"charged"},{"key":"app.payment.transaction.id","type":"string","value":"bbf912fe-0a55-4704-8eb9-02d43f60297d"}]},{"timestamp":1750044449746988,"fields":[{"key":"event","type":"string","value":"shipped"},{"key":"app.shipping.tracking.id","type":"string","value":"4668b5f9-17e2-4311-8b20-c7cf3b08ab39"}]},{"timestamp":1750044449776318,"fields":[{"key":"feature_flag.key","type":"string","value":"kafkaQueueProblems"},{"key":"feature_flag.provider_name","type":"string","value":"flagd"},{"key":"feature_flag.variant","type":"string","value":"off"},{"key":"event","type":"string","value":"feature_flag"}]}],"operationName":"oteldemo.CheckoutService/PlaceOrder","processID":"p3","references":[{"refType":"CHILD_OF","spanID":"b1cf4a62984b9984","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"7683762fa74ffd1c","startTime":1750044449706551,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"app.order.items.count","type":"string","value":"1"},{"key":"app.user.currency","type":"string","value":"USD"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"PlaceOrder"},{"key":"rpc.service","type":"string","value":"oteldemo.CheckoutService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"server.address","type":"string","value":"172.18.0.24"},{"key":"server.port","type":"string","value":"38682"},{"key":"error","type":"string","value":"unset"},{"key":"app.order.amount","type":"string","value":"1102"},{"key":"app.order.id","type":"string","value":"d52a1b43-4a61-11f0-9e2b-96226e8767f9"},{"key":"app.shipping.amount","type":"string","value":"227"},{"key":"app.shipping.tracking.id","type":"string","value":"4668b5f9-17e2-4311-8b20-c7cf3b08ab39"},{"key":"app.user.id","type":"string","value":"d526648e-4a61-11f0-8b6b-b20e5443dfb5"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":19817,"logs":[{"timestamp":1750044449735392,"fields":[{"key":"event","type":"string","value":"Received Quote"},{"key":"app.shipping.cost.total","type":"string","value":"227.50"}]}],"operationName":"/get-quote","processID":"p5","references":[{"refType":"CHILD_OF","spanID":"7b92ebafc9a2a0f1","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"599cbbf8e81ddaca","startTime":1750044449715635,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"opentelemetry-instrumentation-actix-web"},{"key":"otel.scope.version","type":"string","value":"0.22.0"},{"key":"client.address","type":"string","value":"172.18.0.23"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"http.route","type":"string","value":"/get-quote"},{"key":"network.protocol.version","type":"string","value":"1.1"},{"key":"server.address","type":"string","value":"shipping"},{"key":"server.port","type":"string","value":"50050"},{"key":"url.path","type":"string","value":"/get-quote"},{"key":"url.scheme","type":"string","value":"http"},{"key":"user_agent.original","type":"string","value":"Go-http-client/1.1"},{"key":"error","type":"string","value":"unset"},{"key":"app.shipping.cost.total","type":"string","value":"227.50"},{"key":"messaging.message.body.size","type":"string","value":"182"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":1733,"logs":[],"operationName":"grpc.oteldemo.ProductCatalogService/GetProduct","processID":"p4","references":[{"refType":"CHILD_OF","spanID":"6b73da57ebca1b82","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"394722a3d65e5bee","startTime":1750044449777000,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"@opentelemetry/instrumentation-grpc"},{"key":"otel.scope.version","type":"string","value":"0.57.1"},{"key":"net.peer.name","type":"string","value":"product-catalog"},{"key":"net.peer.port","type":"string","value":"3550"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"GetProduct"},{"key":"rpc.service","type":"string","value":"oteldemo.ProductCatalogService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":805,"logs":[],"operationName":"orders publish","processID":"p3","references":[{"refType":"CHILD_OF","spanID":"7683762fa74ffd1c","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"842ad77105e18d23","startTime":1750044449775517,"tags":[{"key":"span.kind","type":"string","value":"producer"},{"key":"otel.scope.name","type":"string","value":"checkout"},{"key":"messaging.destination.name","type":"string","value":"orders"},{"key":"messaging.kafka.destination.partition","type":"string","value":"0"},{"key":"messaging.kafka.message.offset","type":"string","value":"0"},{"key":"messaging.kafka.producer.success","type":"string","value":"true"},{"key":"messaging.operation","type":"string","value":"publish"},{"key":"messaging.system","type":"string","value":"kafka"},{"key":"network.transport","type":"string","value":"tcp"},{"key":"peer.service","type":"string","value":"kafka"},{"key":"error","type":"string","value":"unset"},{"key":"messaging.kafka.producer.duration_ms","type":"string","value":"0"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":22024,"logs":[],"operationName":"HTTP POST","processID":"p3","references":[{"refType":"CHILD_OF","spanID":"96f2298052cc3fda","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"7b92ebafc9a2a0f1","startTime":1750044449713664,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"network.protocol.version","type":"string","value":"1.1"},{"key":"error","type":"string","value":"unset"},{"key":"server.address","type":"string","value":"shipping"},{"key":"server.port","type":"string","value":"50050"},{"key":"url.full","type":"string","value":"http://shipping:50050/get-quote"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":391,"logs":[],"operationName":"HTTP POST","processID":"p3","references":[{"refType":"CHILD_OF","spanID":"7683762fa74ffd1c","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"92345ad5d7cb4190","startTime":1750044449746559,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"network.protocol.version","type":"string","value":"1.1"},{"key":"error","type":"string","value":"unset"},{"key":"server.address","type":"string","value":"shipping"},{"key":"server.port","type":"string","value":"50050"},{"key":"url.full","type":"string","value":"http://shipping:50050/ship-order"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":15663,"logs":[],"operationName":"HTTP POST","processID":"p3","references":[{"refType":"CHILD_OF","spanID":"7683762fa74ffd1c","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"d96adf1246ad7d75","startTime":1750044449759771,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"network.protocol.version","type":"string","value":"1.1"},{"key":"error","type":"string","value":"unset"},{"key":"server.address","type":"string","value":"email"},{"key":"server.port","type":"string","value":"6060"},{"key":"url.full","type":"string","value":"http://email:6060/send_order_confirmation"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":79737,"logs":[],"operationName":"POST","processID":"p8","references":[],"spanID":"10d27d153c44c541","startTime":1750044449700847,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"opentelemetry.instrumentation.requests"},{"key":"otel.scope.version","type":"string","value":"0.55b0"},{"key":"http.method","type":"string","value":"POST"},{"key":"http.status_code","type":"string","value":"200"},{"key":"http.url","type":"string","value":"http://frontend-proxy:8080/api/checkout"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":136,"logs":[{"timestamp":1750044449752991,"fields":[{"key":"message.id","type":"string","value":"1"},{"key":"message.type","type":"string","value":"RECEIVED"},{"key":"event","type":"string","value":"message"},{"key":"message.uncompressed_size","type":"string","value":"15"}]},{"timestamp":1750044449753111,"fields":[{"key":"message.id","type":"string","value":"1"},{"key":"message.type","type":"string","value":"SENT"},{"key":"message.uncompressed_size","type":"string","value":"15"},{"key":"event","type":"string","value":"message"}]}],"operationName":"flagd.evaluation.v1.Service/ResolveBoolean","processID":"p7","references":[{"refType":"CHILD_OF","spanID":"31d9931c1b054f86","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"9d054ff4aeb2b518","startTime":1750044449752984,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"connectrpc.com/otelconnect"},{"key":"otel.scope.version","type":"string","value":"semver:0.6.0-dev"},{"key":"rpc.method","type":"string","value":"ResolveBoolean"},{"key":"rpc.service","type":"string","value":"flagd.evaluation.v1.Service"},{"key":"error","type":"string","value":"unset"},{"key":"net.peer.name","type":"string","value":"172.18.0.10"},{"key":"net.peer.port","type":"string","value":"46838"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.system","type":"string","value":"grpc"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":2157,"logs":[],"operationName":"oteldemo.CurrencyService/Convert","processID":"p3","references":[{"refType":"CHILD_OF","spanID":"96f2298052cc3fda","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"34a9d7aa3afe1688","startTime":1750044449711310,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"Convert"},{"key":"rpc.service","type":"string","value":"oteldemo.CurrencyService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"server.address","type":"string","value":"172.18.0.18"},{"key":"server.port","type":"string","value":"7001"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":2021,"logs":[],"operationName":"oteldemo.CurrencyService/Convert","processID":"p3","references":[{"refType":"CHILD_OF","spanID":"96f2298052cc3fda","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"11295d69d0e661dd","startTime":1750044449735781,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"Convert"},{"key":"rpc.service","type":"string","value":"oteldemo.CurrencyService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"server.address","type":"string","value":"172.18.0.18"},{"key":"server.port","type":"string","value":"7001"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":19397,"logs":[],"operationName":"POST quote","processID":"p5","references":[{"refType":"CHILD_OF","spanID":"599cbbf8e81ddaca","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"09b03b9b5481c29c","startTime":1750044449715774,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"opentelemetry-instrumentation-actix-web"},{"key":"otel.scope.version","type":"string","value":"0.22.0"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"server.address","type":"string","value":"quote"},{"key":"server.port","type":"string","value":"8090"},{"key":"url.full","type":"string","value":"http://quote:8090/getquote"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":78545,"logs":[],"operationName":"router frontend egress","processID":"p9","references":[{"refType":"CHILD_OF","spanID":"d66da216bedd159f","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"df1b3d5c8e0ab6be","startTime":1750044449701376,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"component","type":"string","value":"proxy"},{"key":"http.protocol","type":"string","value":"HTTP/1.1"},{"key":"peer.address","type":"string","value":"172.18.0.24:8080"},{"key":"upstream_address","type":"string","value":"172.18.0.24:8080"},{"key":"upstream_cluster","type":"string","value":"frontend"},{"key":"upstream_cluster.name","type":"string","value":"frontend"},{"key":"error","type":"string","value":"unset"},{"key":"http.status_code","type":"string","value":"200"},{"key":"response_flags","type":"string","value":"-"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":710,"logs":[],"operationName":"oteldemo.ProductCatalogService/GetProduct","processID":"p3","references":[{"refType":"CHILD_OF","spanID":"96f2298052cc3fda","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"7e5e7c2f1ea9cb0b","startTime":1750044449710565,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"GetProduct"},{"key":"rpc.service","type":"string","value":"oteldemo.ProductCatalogService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"server.address","type":"string","value":"172.18.0.19"},{"key":"server.port","type":"string","value":"3550"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":915,"logs":[{"timestamp":1750044449709335,"fields":[{"key":"event","type":"string","value":"Fetch cart"}]}],"operationName":"POST /oteldemo.CartService/GetCart","processID":"p10","references":[{"refType":"CHILD_OF","spanID":"111cb151fdd9a915","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"fefa4832f9254043","startTime":1750044449709238,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"Microsoft.AspNetCore"},{"key":"grpc.method","type":"string","value":"/oteldemo.CartService/GetCart"},{"key":"grpc.status_code","type":"string","value":"0"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"http.route","type":"string","value":"/oteldemo.CartService/GetCart"},{"key":"network.protocol.version","type":"string","value":"2"},{"key":"server.address","type":"string","value":"cart"},{"key":"server.port","type":"string","value":"7070"},{"key":"url.path","type":"string","value":"/oteldemo.CartService/GetCart"},{"key":"url.scheme","type":"string","value":"http"},{"key":"error","type":"string","value":"unset"},{"key":"app.cart.items.count","type":"string","value":"5"},{"key":"app.user.id","type":"string","value":"d526648e-4a61-11f0-8b6b-b20e5443dfb5"},{"key":"user_agent.original","type":"string","value":"grpc-go/1.72.2"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":8349,"logs":[],"operationName":"POST /send_order_confirmation","processID":"p1","references":[{"refType":"CHILD_OF","spanID":"d96adf1246ad7d75","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"739cd04d718779ae","startTime":1750044449766969,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"OpenTelemetry::Instrumentation::Rack"},{"key":"otel.scope.version","type":"string","value":"0.26.0"},{"key":"http.host","type":"string","value":"email:6060"},{"key":"http.method","type":"string","value":"POST"},{"key":"http.route","type":"string","value":"/send_order_confirmation"},{"key":"http.scheme","type":"string","value":"http"},{"key":"http.status_code","type":"string","value":"200"},{"key":"http.target","type":"string","value":"/send_order_confirmation"},{"key":"http.user_agent","type":"string","value":"Go-http-client/1.1"},{"key":"error","type":"string","value":"unset"},{"key":"app.order.id","type":"string","value":"d52a1b43-4a61-11f0-9e2b-96226e8767f9"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":11927,"logs":[{"timestamp":1750044449747830,"fields":[{"key":"event","type":"string","value":"Empty cart"}]},{"timestamp":1750044449755100,"fields":[{"key":"feature_flag.key","type":"string","value":"cartFailure"},{"key":"feature_flag.provider_name","type":"string","value":"flagd Provider"},{"key":"feature_flag.variant","type":"string","value":"off"},{"key":"event","type":"string","value":"feature_flag"}]}],"operationName":"POST /oteldemo.CartService/EmptyCart","processID":"p10","references":[{"refType":"CHILD_OF","spanID":"4e08d386db6de0e6","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"d8802687844ff0da","startTime":1750044449747360,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"Microsoft.AspNetCore"},{"key":"app.user.id","type":"string","value":"d526648e-4a61-11f0-8b6b-b20e5443dfb5"},{"key":"feature_flag.key","type":"string","value":"cartFailure"},{"key":"feature_flag.provider_name","type":"string","value":"flagd Provider"},{"key":"feature_flag.variant","type":"string","value":"off"},{"key":"grpc.method","type":"string","value":"/oteldemo.CartService/EmptyCart"},{"key":"grpc.status_code","type":"string","value":"0"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"http.route","type":"string","value":"/oteldemo.CartService/EmptyCart"},{"key":"network.protocol.version","type":"string","value":"2"},{"key":"server.address","type":"string","value":"cart"},{"key":"server.port","type":"string","value":"7070"},{"key":"url.path","type":"string","value":"/oteldemo.CartService/EmptyCart"},{"key":"url.scheme","type":"string","value":"http"},{"key":"user_agent.original","type":"string","value":"grpc-go/1.72.2"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":74743,"logs":[],"operationName":"grpc.oteldemo.CheckoutService/PlaceOrder","processID":"p4","references":[{"refType":"CHILD_OF","spanID":"6b73da57ebca1b82","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"b1cf4a62984b9984","startTime":1750044449702000,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"@opentelemetry/instrumentation-grpc"},{"key":"otel.scope.version","type":"string","value":"0.57.1"},{"key":"net.peer.name","type":"string","value":"checkout"},{"key":"net.peer.port","type":"string","value":"5050"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"PlaceOrder"},{"key":"rpc.service","type":"string","value":"oteldemo.CheckoutService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":12631,"logs":[],"operationName":"oteldemo.CartService/EmptyCart","processID":"p3","references":[{"refType":"CHILD_OF","spanID":"7683762fa74ffd1c","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"4e08d386db6de0e6","startTime":1750044449747019,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"EmptyCart"},{"key":"rpc.service","type":"string","value":"oteldemo.CartService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"server.address","type":"string","value":"172.18.0.10"},{"key":"server.port","type":"string","value":"7070"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":352,"logs":[{"timestamp":1750044449709386,"fields":[{"key":"event","type":"string","value":"Enqueued"}]},{"timestamp":1750044449709400,"fields":[{"key":"event","type":"string","value":"Sent"}]},{"timestamp":1750044449709718,"fields":[{"key":"event","type":"string","value":"ResponseReceived"}]}],"operationName":"HGET","processID":"p10","references":[{"refType":"CHILD_OF","spanID":"fefa4832f9254043","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"1c6fa81981e4960c","startTime":1750044449709366,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"OpenTelemetry.Instrumentation.StackExchangeRedis"},{"key":"otel.scope.version","type":"string","value":"1.11.0-beta.2"},{"key":"db.redis.database_index","type":"string","value":"0"},{"key":"db.redis.flags","type":"string","value":"None"},{"key":"db.system","type":"string","value":"redis"},{"key":"server.address","type":"string","value":"valkey-cart"},{"key":"server.port","type":"string","value":"6379"},{"key":"error","type":"string","value":"unset"},{"key":"db.statement","type":"string","value":"HGET d526648e-4a61-11f0-8b6b-b20e5443dfb5"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":30309,"logs":[],"operationName":"prepareOrderItemsAndShippingQuoteFromCart","processID":"p3","references":[{"refType":"CHILD_OF","spanID":"7683762fa74ffd1c","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"96f2298052cc3fda","startTime":1750044449707511,"tags":[{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"checkout"},{"key":"app.order.items.count","type":"string","value":"1"},{"key":"error","type":"string","value":"unset"},{"key":"app.cart.items.count","type":"string","value":"5"},{"key":"app.shipping.amount","type":"string","value":"227"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":4711,"logs":[],"operationName":"POST","processID":"p10","references":[{"refType":"CHILD_OF","spanID":"64e503f233846241","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"31d9931c1b054f86","startTime":1750044449749545,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"System.Net.Http"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"network.protocol.version","type":"string","value":"2"},{"key":"server.address","type":"string","value":"flagd"},{"key":"server.port","type":"string","value":"8013"},{"key":"url.full","type":"string","value":"http://flagd:8013/flagd.evaluation.v1.Service/ResolveBoolean"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":3076,"logs":[],"operationName":"grpc.oteldemo.PaymentService/Charge","processID":"p6","references":[{"refType":"CHILD_OF","spanID":"530667cc212dd6ed","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"df89f1712cb9fdec","startTime":1750044449742000,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"@opentelemetry/instrumentation-grpc"},{"key":"otel.scope.version","type":"string","value":"0.57.1"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"Charge"},{"key":"rpc.service","type":"string","value":"oteldemo.PaymentService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"error","type":"string","value":"unset"},{"key":"app.payment.amount","type":"string","value":"1102.50"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":421,"logs":[{"timestamp":1750044449755249,"fields":[{"key":"event","type":"string","value":"Enqueued"}]},{"timestamp":1750044449755262,"fields":[{"key":"event","type":"string","value":"Sent"}]},{"timestamp":1750044449755655,"fields":[{"key":"event","type":"string","value":"ResponseReceived"}]}],"operationName":"HMSET","processID":"p10","references":[{"refType":"CHILD_OF","spanID":"d8802687844ff0da","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"5f78a21a81d1a9a3","startTime":1750044449755233,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"OpenTelemetry.Instrumentation.StackExchangeRedis"},{"key":"otel.scope.version","type":"string","value":"1.11.0-beta.2"},{"key":"db.redis.database_index","type":"string","value":"0"},{"key":"db.redis.flags","type":"string","value":"DemandMaster"},{"key":"db.system","type":"string","value":"redis"},{"key":"server.address","type":"string","value":"valkey-cart"},{"key":"server.port","type":"string","value":"6379"},{"key":"error","type":"string","value":"unset"},{"key":"db.statement","type":"string","value":"HMSET d526648e-4a61-11f0-8b6b-b20e5443dfb5"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":77796,"logs":[],"operationName":"POST /api/checkout","processID":"p4","references":[{"refType":"CHILD_OF","spanID":"47c48aa63a0c5a3d","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"01468af9419620f5","startTime":1750044449701000,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"next.js"},{"key":"otel.scope.version","type":"string","value":"0.0.1"},{"key":"http.method","type":"string","value":"POST"},{"key":"http.status_code","type":"string","value":"200"},{"key":"http.target","type":"string","value":"/api/checkout"},{"key":"next.rsc","type":"string","value":"false"},{"key":"next.span_name","type":"string","value":"POST /api/checkout"},{"key":"next.span_type","type":"string","value":"BaseServer.handleRequest"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":5855,"logs":[],"operationName":"flagd.evaluation.v1.Service/ResolveBoolean","processID":"p10","references":[{"refType":"CHILD_OF","spanID":"d8802687844ff0da","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"64e503f233846241","startTime":1750044449749012,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"OpenTelemetry.Instrumentation.GrpcNetClient"},{"key":"otel.scope.version","type":"string","value":"1.11.0-beta.2"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"ResolveBoolean"},{"key":"rpc.service","type":"string","value":"flagd.evaluation.v1.Service"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"server.address","type":"string","value":"flagd"},{"key":"server.port","type":"string","value":"8013"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":877,"logs":[{"timestamp":1750044449755696,"fields":[{"key":"event","type":"string","value":"Enqueued"}]},{"timestamp":1750044449755708,"fields":[{"key":"event","type":"string","value":"Sent"}]},{"timestamp":1750044449756563,"fields":[{"key":"event","type":"string","value":"ResponseReceived"}]}],"operationName":"EXPIRE","processID":"p10","references":[{"refType":"CHILD_OF","spanID":"d8802687844ff0da","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"4a42b7a5fa81bdfb","startTime":1750044449755686,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"OpenTelemetry.Instrumentation.StackExchangeRedis"},{"key":"otel.scope.version","type":"string","value":"1.11.0-beta.2"},{"key":"db.redis.database_index","type":"string","value":"0"},{"key":"db.redis.flags","type":"string","value":"DemandMaster"},{"key":"db.system","type":"string","value":"redis"},{"key":"server.address","type":"string","value":"valkey-cart"},{"key":"server.port","type":"string","value":"6379"},{"key":"error","type":"string","value":"unset"},{"key":"db.statement","type":"string","value":"EXPIRE d526648e-4a61-11f0-8b6b-b20e5443dfb5"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":75,"logs":[{"timestamp":1750044449711020,"fields":[{"key":"event","type":"string","value":"Product Found"}]}],"operationName":"oteldemo.ProductCatalogService/GetProduct","processID":"p11","references":[{"refType":"CHILD_OF","spanID":"7e5e7c2f1ea9cb0b","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"5b997902f830009b","startTime":1750044449710969,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"GetProduct"},{"key":"rpc.service","type":"string","value":"oteldemo.ProductCatalogService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"error","type":"string","value":"unset"},{"key":"app.product.id","type":"string","value":"0PUK6V6EV0"},{"key":"app.product.name","type":"string","value":"Solar System Color Imager"},{"key":"server.address","type":"string","value":"172.18.0.23"},{"key":"server.port","type":"string","value":"56058"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":78,"logs":[{"timestamp":1750044449778775,"fields":[{"key":"event","type":"string","value":"Product Found"}]}],"operationName":"oteldemo.ProductCatalogService/GetProduct","processID":"p11","references":[{"refType":"CHILD_OF","spanID":"394722a3d65e5bee","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"212f00429ff724f5","startTime":1750044449778734,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"GetProduct"},{"key":"rpc.service","type":"string","value":"oteldemo.ProductCatalogService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"error","type":"string","value":"unset"},{"key":"app.product.id","type":"string","value":"0PUK6V6EV0"},{"key":"app.product.name","type":"string","value":"Solar System Color Imager"},{"key":"server.address","type":"string","value":"172.18.0.24"},{"key":"server.port","type":"string","value":"47538"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":597,"logs":[{"timestamp":1750044449711719,"fields":[{"key":"event","type":"string","value":"Processing currency conversion request"}]},{"timestamp":1750044449711741,"fields":[{"key":"event","type":"string","value":"Conversion successful, response sent back"}]}],"operationName":"Currency/Convert","processID":"p12","references":[{"refType":"CHILD_OF","spanID":"34a9d7aa3afe1688","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"42e4324fcb045b99","startTime":1750044449711715,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"currency"},{"key":"app.currency.conversion.from","type":"string","value":"USD"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"Convert"},{"key":"rpc.service","type":"string","value":"oteldemo.CurrencyService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"error","type":"string","value":"false"},{"key":"app.currency.conversion.to","type":"string","value":"USD"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":655,"logs":[{"timestamp":1750044449736390,"fields":[{"key":"event","type":"string","value":"Processing currency conversion request"}]},{"timestamp":1750044449736414,"fields":[{"key":"event","type":"string","value":"Conversion successful, response sent back"}]}],"operationName":"Currency/Convert","processID":"p12","references":[{"refType":"CHILD_OF","spanID":"11295d69d0e661dd","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"adb556f3c99b633d","startTime":1750044449736386,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"currency"},{"key":"app.currency.conversion.from","type":"string","value":"USD"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"Convert"},{"key":"rpc.service","type":"string","value":"oteldemo.CurrencyService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"error","type":"string","value":"false"},{"key":"app.currency.conversion.to","type":"string","value":"USD"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":78648,"logs":[],"operationName":"ingress","processID":"p9","references":[{"refType":"CHILD_OF","spanID":"10d27d153c44c541","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"d66da216bedd159f","startTime":1750044449701298,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"component","type":"string","value":"proxy"},{"key":"downstream_cluster","type":"string","value":"-"},{"key":"http.protocol","type":"string","value":"HTTP/1.1"},{"key":"node_id","type":"string","value":"-"},{"key":"peer.address","type":"string","value":"172.18.0.25"},{"key":"zone","type":"string","value":"-"},{"key":"guid:x-request-id","type":"string","value":"347edd6d-e273-953e-87f6-7ba07f352331"},{"key":"http.method","type":"string","value":"POST"},{"key":"http.status_code","type":"string","value":"200"},{"key":"http.url","type":"string","value":"http://frontend-proxy:8080/api/checkout"},{"key":"request_size","type":"string","value":"388"},{"key":"response_flags","type":"string","value":"-"},{"key":"response_size","type":"string","value":"857"},{"key":"upstream_cluster","type":"string","value":"frontend"},{"key":"upstream_cluster.name","type":"string","value":"frontend"},{"key":"user_agent","type":"string","value":"python-requests/2.32.4"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null}],"errors":null,"limit":0,"offset":0,"total":1}
    

Find a trace by trace_id

      curl http://<victoria-traces>:9428/select/jaeger/api/traces/9e06226196051d9c3c10dfab343791ad

Here’s a response example:

      {"data":[{"processes":{"p1":{"serviceName":"email","tags":[{"key":"process.command","type":"string","value":"email_server.rb"},{"key":"process.pid","type":"string","value":"1"},{"key":"process.runtime.description","type":"string","value":"ruby 3.4.4 (2025-05-14 revision a38531fd3f) +PRISM [aarch64-linux-musl]"},{"key":"process.runtime.name","type":"string","value":"ruby"},{"key":"process.runtime.version","type":"string","value":"3.4.4"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"ruby"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.8.0"}]},"p10":{"serviceName":"load-generator","tags":[{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"python"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.34.0"}]},"p11":{"serviceName":"product-catalog","tags":[{"key":"host.name","type":"string","value":"3dabfcfe8381"},{"key":"os.description","type":"string","value":"Debian GNU/Linux Debian GNU/Linux 12 (bookworm) (Linux 3dabfcfe8381 6.10.14-linuxkit #1 SMP Tue Apr 15 16:00:54 UTC 2025 aarch64)"},{"key":"os.type","type":"string","value":"linux"},{"key":"process.command_args","type":"string","value":"[\"./product-catalog\"]"},{"key":"process.executable.name","type":"string","value":"product-catalog"},{"key":"process.executable.path","type":"string","value":"/usr/src/app/product-catalog"},{"key":"process.owner","type":"string","value":"nonroot"},{"key":"process.pid","type":"string","value":"1"},{"key":"process.runtime.description","type":"string","value":"go version go1.24.4 linux/arm64"},{"key":"process.runtime.name","type":"string","value":"go"},{"key":"process.runtime.version","type":"string","value":"go1.24.4"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"go"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.36.0"}]},"p12":{"serviceName":"currency","tags":[{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"cpp"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.20.0"}]},"p2":{"serviceName":"quote","tags":[{"key":"container.id","type":"string","value":"759183873eeb1328f16df8ea5b5a10932506af136a6537c6a365131c04f1645c"},{"key":"host.arch","type":"string","value":"aarch64"},{"key":"host.name","type":"string","value":"759183873eeb"},{"key":"os.description","type":"string","value":"6.10.14-linuxkit"},{"key":"os.name","type":"string","value":"Linux"},{"key":"os.type","type":"string","value":"linux"},{"key":"os.version","type":"string","value":"#1 SMP Tue Apr 15 16:00:54 UTC 2025"},{"key":"process.command","type":"string","value":"public/index.php"},{"key":"process.command_args","type":"string","value":"[\"public/index.php\"]"},{"key":"process.executable.path","type":"string","value":"/usr/local/bin/php"},{"key":"process.owner","type":"string","value":"www-data"},{"key":"process.pid","type":"string","value":"1"},{"key":"process.runtime.name","type":"string","value":"cli"},{"key":"process.runtime.version","type":"string","value":"8.3.22"},{"key":"service.instance.id","type":"string","value":"9dc0abaa-c408-483e-9fed-8375a73efb91"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.distro.name","type":"string","value":"opentelemetry-php-instrumentation"},{"key":"telemetry.distro.version","type":"string","value":"1.1.3"},{"key":"telemetry.sdk.language","type":"string","value":"php"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.5.0"}]},"p3":{"serviceName":"frontend","tags":[{"key":"container.id","type":"string","value":"2d395f01353040612a00252cf6e8c32f00ab94ae06f82f143a3ea9c742072674"},{"key":"host.arch","type":"string","value":"arm64"},{"key":"host.name","type":"string","value":"2d395f013530"},{"key":"os.type","type":"string","value":"linux"},{"key":"os.version","type":"string","value":"6.10.14-linuxkit"},{"key":"process.command","type":"string","value":"/app/server.js"},{"key":"process.command_args","type":"string","value":"[\"/usr/local/bin/node\",\"--require\",\"./Instrumentation.js\",\"/app/server.js\"]"},{"key":"process.executable.name","type":"string","value":"node"},{"key":"process.executable.path","type":"string","value":"/usr/local/bin/node"},{"key":"process.owner","type":"string","value":"nextjs"},{"key":"process.pid","type":"string","value":"17"},{"key":"process.runtime.description","type":"string","value":"Node.js"},{"key":"process.runtime.name","type":"string","value":"nodejs"},{"key":"process.runtime.version","type":"string","value":"22.16.0"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"nodejs"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.30.1"}]},"p4":{"serviceName":"payment","tags":[{"key":"container.id","type":"string","value":"18ee03279d38ed0e0eedad037c260df78dfc3323aa662ca14a2d38fcc8bf3762"},{"key":"host.arch","type":"string","value":"arm64"},{"key":"host.name","type":"string","value":"18ee03279d38"},{"key":"os.type","type":"string","value":"linux"},{"key":"os.version","type":"string","value":"6.10.14-linuxkit"},{"key":"process.command","type":"string","value":"/usr/src/app/index.js"},{"key":"process.command_args","type":"string","value":"[\"/usr/local/bin/node\",\"--require\",\"./opentelemetry.js\",\"/usr/src/app/index.js\"]"},{"key":"process.executable.name","type":"string","value":"node"},{"key":"process.executable.path","type":"string","value":"/usr/local/bin/node"},{"key":"process.owner","type":"string","value":"node"},{"key":"process.pid","type":"string","value":"17"},{"key":"process.runtime.description","type":"string","value":"Node.js"},{"key":"process.runtime.name","type":"string","value":"nodejs"},{"key":"process.runtime.version","type":"string","value":"22.16.0"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"nodejs"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.30.1"}]},"p5":{"serviceName":"flagd","tags":[{"key":"host.name","type":"string","value":"1f315d8a0f78"},{"key":"os.description","type":"string","value":"Debian GNU/Linux Debian GNU/Linux 12 (bookworm) (Linux 1f315d8a0f78 6.10.14-linuxkit #1 SMP Tue Apr 15 16:00:54 UTC 2025 aarch64)"},{"key":"os.type","type":"string","value":"linux"},{"key":"process.runtime.version","type":"string","value":"go1.24.1"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"v0.12.3"},{"key":"telemetry.sdk.language","type":"string","value":"go"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.35.0"}]},"p6":{"serviceName":"shipping","tags":[{"key":"os.type","type":"string","value":"linux"},{"key":"process.command_args","type":"string","value":"[\"/app/shipping\"]"},{"key":"process.pid","type":"string","value":"1"},{"key":"process.runtime.description","type":"string","value":"rustc 1.82.0 (f6e511eec 2024-10-15)"},{"key":"process.runtime.name","type":"string","value":"rustc"},{"key":"process.runtime.version","type":"string","value":"1.82.0"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"rust"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"0.30.0"}]},"p7":{"serviceName":"checkout","tags":[{"key":"host.name","type":"string","value":"cbdb5e0808c2"},{"key":"os.description","type":"string","value":"Debian GNU/Linux Debian GNU/Linux 12 (bookworm) (Linux cbdb5e0808c2 6.10.14-linuxkit #1 SMP Tue Apr 15 16:00:54 UTC 2025 aarch64)"},{"key":"os.type","type":"string","value":"linux"},{"key":"process.command_args","type":"string","value":"[\"./checkout\"]"},{"key":"process.executable.name","type":"string","value":"checkout"},{"key":"process.executable.path","type":"string","value":"/usr/src/app/checkout"},{"key":"process.owner","type":"string","value":"nonroot"},{"key":"process.pid","type":"string","value":"1"},{"key":"process.runtime.description","type":"string","value":"go version go1.24.4 linux/arm64"},{"key":"process.runtime.name","type":"string","value":"go"},{"key":"process.runtime.version","type":"string","value":"go1.24.4"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"go"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.36.0"}]},"p8":{"serviceName":"frontend-proxy","tags":[{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"}]},"p9":{"serviceName":"cart","tags":[{"key":"container.id","type":"string","value":"5603ff989877ecf311403b6ea81fda10734846a0cbdad3a09c39fb068e4a07fc"},{"key":"host.name","type":"string","value":"5603ff989877"},{"key":"service.namespace","type":"string","value":"opentelemetry-demo"},{"key":"service.version","type":"string","value":"2.0.2"},{"key":"telemetry.sdk.language","type":"string","value":"dotnet"},{"key":"telemetry.sdk.name","type":"string","value":"opentelemetry"},{"key":"telemetry.sdk.version","type":"string","value":"1.11.2"}]}},"spans":[{"duration":4935,"logs":[],"operationName":"send_email","processID":"p1","references":[{"refType":"CHILD_OF","spanID":"739cd04d718779ae","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"032bf7007e123e8d","startTime":1750044449769690,"tags":[{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"email"},{"key":"error","type":"string","value":"unset"},{"key":"app.email.recipient","type":"string","value":"reed@example.com"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":3339,"logs":[{"timestamp":1750044449717803,"fields":[{"key":"event","type":"string","value":"Received get quote request, processing it"}]},{"timestamp":1750044449718100,"fields":[{"key":"event","type":"string","value":"Quote processed, response sent back"},{"key":"app.quote.cost.total","type":"string","value":"227.5"}]}],"operationName":"{closure}","processID":"p2","references":[{"refType":"CHILD_OF","spanID":"aaf29afb62662d95","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"ea80042fbe6e5887","startTime":1750044449717692,"tags":[{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"io.opentelemetry.contrib.php.slim"},{"key":"code.file.path","type":"string","value":"/var/www/vendor/php-di/slim-bridge/src/ControllerInvoker.php"},{"key":"code.function.name","type":"string","value":"DI\\Bridge\\Slim\\ControllerInvoker::__invoke"},{"key":"code.line.number","type":"string","value":"29"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":6544,"logs":[],"operationName":"POST /getquote","processID":"p2","references":[{"refType":"CHILD_OF","spanID":"09b03b9b5481c29c","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"aaf29afb62662d95","startTime":1750044449717102,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"io.opentelemetry.contrib.php.slim"},{"key":"code.file.path","type":"string","value":"/var/www/vendor/slim/slim/Slim/App.php"},{"key":"code.function.name","type":"string","value":"Slim\\App::handle"},{"key":"code.line.number","type":"string","value":"207"},{"key":"http.request.body.size","type":"string","value":"19"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.body.size","type":"string","value":"-"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"http.route","type":"string","value":"/getquote"},{"key":"network.protocol.version","type":"string","value":"1.1"},{"key":"server.address","type":"string","value":"quote"},{"key":"server.port","type":"string","value":"8090"},{"key":"url.full","type":"string","value":"http://quote:8090/getquote"},{"key":"url.path","type":"string","value":"/getquote"},{"key":"url.scheme","type":"string","value":"http"},{"key":"user_agent.original","type":"string","value":"-"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":77220,"logs":[],"operationName":"executing api route (pages) /api/checkout","processID":"p3","references":[{"refType":"CHILD_OF","spanID":"01468af9419620f5","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"6b73da57ebca1b82","startTime":1750044449702000,"tags":[{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"next.js"},{"key":"otel.scope.version","type":"string","value":"0.0.1"},{"key":"http.status_code","type":"string","value":"200"},{"key":"next.span_name","type":"string","value":"executing api route (pages) /api/checkout"},{"key":"next.span_type","type":"string","value":"Node.runHandler"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":78153,"logs":[],"operationName":"POST","processID":"p3","references":[{"refType":"CHILD_OF","spanID":"df1b3d5c8e0ab6be","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"47c48aa63a0c5a3d","startTime":1750044449701000,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"@opentelemetry/instrumentation-http"},{"key":"otel.scope.version","type":"string","value":"0.57.1"},{"key":"http.flavor","type":"string","value":"1.1"},{"key":"http.host","type":"string","value":"frontend-proxy:8080"},{"key":"http.method","type":"string","value":"POST"},{"key":"http.scheme","type":"string","value":"http"},{"key":"http.status_code","type":"string","value":"200"},{"key":"http.user_agent","type":"string","value":"python-requests/2.32.4"},{"key":"net.host.name","type":"string","value":"frontend-proxy"},{"key":"net.peer.ip","type":"string","value":"172.18.0.26"},{"key":"net.transport","type":"string","value":"ip_tcp"},{"key":"error","type":"string","value":"unset"},{"key":"http.request_content_length_uncompressed","type":"string","value":"388"},{"key":"http.status_text","type":"string","value":"OK"},{"key":"http.target","type":"string","value":"/api/checkout"},{"key":"http.url","type":"string","value":"http://frontend-proxy:8080/api/checkout"},{"key":"net.host.ip","type":"string","value":"172.18.0.24"},{"key":"net.host.port","type":"string","value":"8080"},{"key":"net.peer.port","type":"string","value":"35632"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":1988,"logs":[],"operationName":"charge","processID":"p4","references":[{"refType":"CHILD_OF","spanID":"df89f1712cb9fdec","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"f30e92001c694787","startTime":1750044449743000,"tags":[{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"payment"},{"key":"app.payment.card_type","type":"string","value":"visa"},{"key":"app.payment.card_valid","type":"string","value":"true"},{"key":"app.payment.charged","type":"string","value":"false"},{"key":"error","type":"string","value":"unset"},{"key":"app.loyalty.level","type":"string","value":"silver"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":6,"logs":[],"operationName":"resolveBoolean","processID":"p5","references":[{"refType":"CHILD_OF","spanID":"3af2ca071042ef47","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"ab8c870e76bbe57f","startTime":1750044449753032,"tags":[{"key":"error","type":"string","value":"unset"},{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"jsonEvaluator"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":70,"logs":[],"operationName":"resolveBoolean","processID":"p5","references":[{"refType":"CHILD_OF","spanID":"9d054ff4aeb2b518","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"3af2ca071042ef47","startTime":1750044449753027,"tags":[{"key":"error","type":"string","value":"unset"},{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"flagd.evaluation.v1"},{"key":"feature_flag.key","type":"string","value":"cartFailure"},{"key":"feature_flag.provider_name","type":"string","value":"flagd"},{"key":"feature_flag.variant","type":"string","value":"off"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":19817,"logs":[{"timestamp":1750044449735392,"fields":[{"key":"event","type":"string","value":"Received Quote"},{"key":"app.shipping.cost.total","type":"string","value":"227.50"}]}],"operationName":"/get-quote","processID":"p6","references":[{"refType":"CHILD_OF","spanID":"7b92ebafc9a2a0f1","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"599cbbf8e81ddaca","startTime":1750044449715635,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"opentelemetry-instrumentation-actix-web"},{"key":"otel.scope.version","type":"string","value":"0.22.0"},{"key":"client.address","type":"string","value":"172.18.0.23"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"http.route","type":"string","value":"/get-quote"},{"key":"network.protocol.version","type":"string","value":"1.1"},{"key":"server.address","type":"string","value":"shipping"},{"key":"server.port","type":"string","value":"50050"},{"key":"url.path","type":"string","value":"/get-quote"},{"key":"url.scheme","type":"string","value":"http"},{"key":"user_agent.original","type":"string","value":"Go-http-client/1.1"},{"key":"error","type":"string","value":"unset"},{"key":"app.shipping.cost.total","type":"string","value":"227.50"},{"key":"messaging.message.body.size","type":"string","value":"182"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":283,"logs":[],"operationName":"sinatra.render_template","processID":"p1","references":[{"refType":"CHILD_OF","spanID":"1fd5f529c2dd316b","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"bc5f262c2f7d9bb5","startTime":1750044449770317,"tags":[{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"OpenTelemetry::Instrumentation::Sinatra"},{"key":"otel.scope.version","type":"string","value":"0.25.0"},{"key":"error","type":"string","value":"unset"},{"key":"sinatra.template_name","type":"string","value":"layout"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":961,"logs":[],"operationName":"sinatra.render_template","processID":"p1","references":[{"refType":"CHILD_OF","spanID":"032bf7007e123e8d","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"1fd5f529c2dd316b","startTime":1750044449769761,"tags":[{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"OpenTelemetry::Instrumentation::Sinatra"},{"key":"otel.scope.version","type":"string","value":"0.25.0"},{"key":"error","type":"string","value":"unset"},{"key":"sinatra.template_name","type":"string","value":"confirmation"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":6755,"logs":[],"operationName":"oteldemo.PaymentService/Charge","processID":"p7","references":[{"refType":"CHILD_OF","spanID":"7683762fa74ffd1c","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"530667cc212dd6ed","startTime":1750044449739280,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"Charge"},{"key":"rpc.service","type":"string","value":"oteldemo.PaymentService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"server.address","type":"string","value":"172.18.0.14"},{"key":"server.port","type":"string","value":"50051"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":1831,"logs":[],"operationName":"oteldemo.CartService/GetCart","processID":"p7","references":[{"refType":"CHILD_OF","spanID":"96f2298052cc3fda","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"111cb151fdd9a915","startTime":1750044449708652,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"GetCart"},{"key":"rpc.service","type":"string","value":"oteldemo.CartService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"server.address","type":"string","value":"172.18.0.10"},{"key":"server.port","type":"string","value":"7070"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":46,"logs":[],"operationName":"/ship-order","processID":"p6","references":[{"refType":"CHILD_OF","spanID":"92345ad5d7cb4190","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"d1253691f90f5b95","startTime":1750044449746781,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"opentelemetry-instrumentation-actix-web"},{"key":"otel.scope.version","type":"string","value":"0.22.0"},{"key":"client.address","type":"string","value":"172.18.0.23"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"http.route","type":"string","value":"/ship-order"},{"key":"network.protocol.version","type":"string","value":"1.1"},{"key":"server.address","type":"string","value":"shipping"},{"key":"server.port","type":"string","value":"50050"},{"key":"url.path","type":"string","value":"/ship-order"},{"key":"url.scheme","type":"string","value":"http"},{"key":"user_agent.original","type":"string","value":"Go-http-client/1.1"},{"key":"error","type":"string","value":"unset"},{"key":"messaging.message.body.size","type":"string","value":"182"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":128,"logs":[{"timestamp":1750044449717887,"fields":[{"key":"event","type":"string","value":"Calculating quote"}]},{"timestamp":1750044449717919,"fields":[{"key":"event","type":"string","value":"Quote calculated, returning its value"}]}],"operationName":"calculate-quote","processID":"p2","references":[{"refType":"CHILD_OF","spanID":"ea80042fbe6e5887","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"0b119b964828c67b","startTime":1750044449717886,"tags":[{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"manual-instrumentation"},{"key":"error","type":"string","value":"unset"},{"key":"app.quote.cost.total","type":"string","value":"227.5"},{"key":"app.quote.items.count","type":"string","value":"5"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":78545,"logs":[],"operationName":"router frontend egress","processID":"p8","references":[{"refType":"CHILD_OF","spanID":"d66da216bedd159f","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"df1b3d5c8e0ab6be","startTime":1750044449701376,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"component","type":"string","value":"proxy"},{"key":"http.protocol","type":"string","value":"HTTP/1.1"},{"key":"peer.address","type":"string","value":"172.18.0.24:8080"},{"key":"upstream_address","type":"string","value":"172.18.0.24:8080"},{"key":"upstream_cluster","type":"string","value":"frontend"},{"key":"upstream_cluster.name","type":"string","value":"frontend"},{"key":"error","type":"string","value":"unset"},{"key":"http.status_code","type":"string","value":"200"},{"key":"response_flags","type":"string","value":"-"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":915,"logs":[{"timestamp":1750044449709335,"fields":[{"key":"event","type":"string","value":"Fetch cart"}]}],"operationName":"POST /oteldemo.CartService/GetCart","processID":"p9","references":[{"refType":"CHILD_OF","spanID":"111cb151fdd9a915","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"fefa4832f9254043","startTime":1750044449709238,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"Microsoft.AspNetCore"},{"key":"grpc.method","type":"string","value":"/oteldemo.CartService/GetCart"},{"key":"grpc.status_code","type":"string","value":"0"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"http.route","type":"string","value":"/oteldemo.CartService/GetCart"},{"key":"network.protocol.version","type":"string","value":"2"},{"key":"server.address","type":"string","value":"cart"},{"key":"server.port","type":"string","value":"7070"},{"key":"url.path","type":"string","value":"/oteldemo.CartService/GetCart"},{"key":"url.scheme","type":"string","value":"http"},{"key":"error","type":"string","value":"unset"},{"key":"app.cart.items.count","type":"string","value":"5"},{"key":"app.user.id","type":"string","value":"d526648e-4a61-11f0-8b6b-b20e5443dfb5"},{"key":"user_agent.original","type":"string","value":"grpc-go/1.72.2"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":710,"logs":[],"operationName":"oteldemo.ProductCatalogService/GetProduct","processID":"p7","references":[{"refType":"CHILD_OF","spanID":"96f2298052cc3fda","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"7e5e7c2f1ea9cb0b","startTime":1750044449710565,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"GetProduct"},{"key":"rpc.service","type":"string","value":"oteldemo.ProductCatalogService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"server.address","type":"string","value":"172.18.0.19"},{"key":"server.port","type":"string","value":"3550"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":69871,"logs":[{"timestamp":1750044449737830,"fields":[{"key":"event","type":"string","value":"prepared"}]},{"timestamp":1750044449739261,"fields":[{"key":"feature_flag.key","type":"string","value":"paymentUnreachable"},{"key":"feature_flag.provider_name","type":"string","value":"flagd"},{"key":"feature_flag.variant","type":"string","value":"off"},{"key":"event","type":"string","value":"feature_flag"}]},{"timestamp":1750044449746517,"fields":[{"key":"event","type":"string","value":"charged"},{"key":"app.payment.transaction.id","type":"string","value":"bbf912fe-0a55-4704-8eb9-02d43f60297d"}]},{"timestamp":1750044449746988,"fields":[{"key":"event","type":"string","value":"shipped"},{"key":"app.shipping.tracking.id","type":"string","value":"4668b5f9-17e2-4311-8b20-c7cf3b08ab39"}]},{"timestamp":1750044449776318,"fields":[{"key":"feature_flag.key","type":"string","value":"kafkaQueueProblems"},{"key":"feature_flag.provider_name","type":"string","value":"flagd"},{"key":"feature_flag.variant","type":"string","value":"off"},{"key":"event","type":"string","value":"feature_flag"}]}],"operationName":"oteldemo.CheckoutService/PlaceOrder","processID":"p7","references":[{"refType":"CHILD_OF","spanID":"b1cf4a62984b9984","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"7683762fa74ffd1c","startTime":1750044449706551,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"app.order.items.count","type":"string","value":"1"},{"key":"app.user.currency","type":"string","value":"USD"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"PlaceOrder"},{"key":"rpc.service","type":"string","value":"oteldemo.CheckoutService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"server.address","type":"string","value":"172.18.0.24"},{"key":"server.port","type":"string","value":"38682"},{"key":"error","type":"string","value":"unset"},{"key":"app.order.amount","type":"string","value":"1102"},{"key":"app.order.id","type":"string","value":"d52a1b43-4a61-11f0-9e2b-96226e8767f9"},{"key":"app.shipping.amount","type":"string","value":"227"},{"key":"app.shipping.tracking.id","type":"string","value":"4668b5f9-17e2-4311-8b20-c7cf3b08ab39"},{"key":"app.user.id","type":"string","value":"d526648e-4a61-11f0-8b6b-b20e5443dfb5"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":8349,"logs":[],"operationName":"POST /send_order_confirmation","processID":"p1","references":[{"refType":"CHILD_OF","spanID":"d96adf1246ad7d75","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"739cd04d718779ae","startTime":1750044449766969,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"OpenTelemetry::Instrumentation::Rack"},{"key":"otel.scope.version","type":"string","value":"0.26.0"},{"key":"http.host","type":"string","value":"email:6060"},{"key":"http.method","type":"string","value":"POST"},{"key":"http.route","type":"string","value":"/send_order_confirmation"},{"key":"http.scheme","type":"string","value":"http"},{"key":"http.status_code","type":"string","value":"200"},{"key":"http.target","type":"string","value":"/send_order_confirmation"},{"key":"http.user_agent","type":"string","value":"Go-http-client/1.1"},{"key":"error","type":"string","value":"unset"},{"key":"app.order.id","type":"string","value":"d52a1b43-4a61-11f0-9e2b-96226e8767f9"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":74743,"logs":[],"operationName":"grpc.oteldemo.CheckoutService/PlaceOrder","processID":"p3","references":[{"refType":"CHILD_OF","spanID":"6b73da57ebca1b82","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"b1cf4a62984b9984","startTime":1750044449702000,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"@opentelemetry/instrumentation-grpc"},{"key":"otel.scope.version","type":"string","value":"0.57.1"},{"key":"net.peer.name","type":"string","value":"checkout"},{"key":"net.peer.port","type":"string","value":"5050"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"PlaceOrder"},{"key":"rpc.service","type":"string","value":"oteldemo.CheckoutService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":12631,"logs":[],"operationName":"oteldemo.CartService/EmptyCart","processID":"p7","references":[{"refType":"CHILD_OF","spanID":"7683762fa74ffd1c","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"4e08d386db6de0e6","startTime":1750044449747019,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"EmptyCart"},{"key":"rpc.service","type":"string","value":"oteldemo.CartService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"server.address","type":"string","value":"172.18.0.10"},{"key":"server.port","type":"string","value":"7070"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":11927,"logs":[{"timestamp":1750044449747830,"fields":[{"key":"event","type":"string","value":"Empty cart"}]},{"timestamp":1750044449755100,"fields":[{"key":"feature_flag.key","type":"string","value":"cartFailure"},{"key":"feature_flag.provider_name","type":"string","value":"flagd Provider"},{"key":"feature_flag.variant","type":"string","value":"off"},{"key":"event","type":"string","value":"feature_flag"}]}],"operationName":"POST /oteldemo.CartService/EmptyCart","processID":"p9","references":[{"refType":"CHILD_OF","spanID":"4e08d386db6de0e6","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"d8802687844ff0da","startTime":1750044449747360,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"Microsoft.AspNetCore"},{"key":"app.user.id","type":"string","value":"d526648e-4a61-11f0-8b6b-b20e5443dfb5"},{"key":"feature_flag.key","type":"string","value":"cartFailure"},{"key":"feature_flag.provider_name","type":"string","value":"flagd Provider"},{"key":"feature_flag.variant","type":"string","value":"off"},{"key":"grpc.method","type":"string","value":"/oteldemo.CartService/EmptyCart"},{"key":"grpc.status_code","type":"string","value":"0"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"http.route","type":"string","value":"/oteldemo.CartService/EmptyCart"},{"key":"network.protocol.version","type":"string","value":"2"},{"key":"server.address","type":"string","value":"cart"},{"key":"server.port","type":"string","value":"7070"},{"key":"url.path","type":"string","value":"/oteldemo.CartService/EmptyCart"},{"key":"url.scheme","type":"string","value":"http"},{"key":"user_agent.original","type":"string","value":"grpc-go/1.72.2"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":1733,"logs":[],"operationName":"grpc.oteldemo.ProductCatalogService/GetProduct","processID":"p3","references":[{"refType":"CHILD_OF","spanID":"6b73da57ebca1b82","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"394722a3d65e5bee","startTime":1750044449777000,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"@opentelemetry/instrumentation-grpc"},{"key":"otel.scope.version","type":"string","value":"0.57.1"},{"key":"net.peer.name","type":"string","value":"product-catalog"},{"key":"net.peer.port","type":"string","value":"3550"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"GetProduct"},{"key":"rpc.service","type":"string","value":"oteldemo.ProductCatalogService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":30309,"logs":[],"operationName":"prepareOrderItemsAndShippingQuoteFromCart","processID":"p7","references":[{"refType":"CHILD_OF","spanID":"7683762fa74ffd1c","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"96f2298052cc3fda","startTime":1750044449707511,"tags":[{"key":"span.kind","type":"string","value":"internal"},{"key":"otel.scope.name","type":"string","value":"checkout"},{"key":"app.order.items.count","type":"string","value":"1"},{"key":"error","type":"string","value":"unset"},{"key":"app.cart.items.count","type":"string","value":"5"},{"key":"app.shipping.amount","type":"string","value":"227"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":805,"logs":[],"operationName":"orders publish","processID":"p7","references":[{"refType":"CHILD_OF","spanID":"7683762fa74ffd1c","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"842ad77105e18d23","startTime":1750044449775517,"tags":[{"key":"span.kind","type":"string","value":"producer"},{"key":"otel.scope.name","type":"string","value":"checkout"},{"key":"messaging.destination.name","type":"string","value":"orders"},{"key":"messaging.kafka.destination.partition","type":"string","value":"0"},{"key":"messaging.kafka.message.offset","type":"string","value":"0"},{"key":"messaging.kafka.producer.success","type":"string","value":"true"},{"key":"messaging.operation","type":"string","value":"publish"},{"key":"messaging.system","type":"string","value":"kafka"},{"key":"network.transport","type":"string","value":"tcp"},{"key":"peer.service","type":"string","value":"kafka"},{"key":"error","type":"string","value":"unset"},{"key":"messaging.kafka.producer.duration_ms","type":"string","value":"0"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":352,"logs":[{"timestamp":1750044449709386,"fields":[{"key":"event","type":"string","value":"Enqueued"}]},{"timestamp":1750044449709400,"fields":[{"key":"event","type":"string","value":"Sent"}]},{"timestamp":1750044449709718,"fields":[{"key":"event","type":"string","value":"ResponseReceived"}]}],"operationName":"HGET","processID":"p9","references":[{"refType":"CHILD_OF","spanID":"fefa4832f9254043","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"1c6fa81981e4960c","startTime":1750044449709366,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"OpenTelemetry.Instrumentation.StackExchangeRedis"},{"key":"otel.scope.version","type":"string","value":"1.11.0-beta.2"},{"key":"db.redis.database_index","type":"string","value":"0"},{"key":"db.redis.flags","type":"string","value":"None"},{"key":"db.system","type":"string","value":"redis"},{"key":"server.address","type":"string","value":"valkey-cart"},{"key":"server.port","type":"string","value":"6379"},{"key":"error","type":"string","value":"unset"},{"key":"db.statement","type":"string","value":"HGET d526648e-4a61-11f0-8b6b-b20e5443dfb5"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":22024,"logs":[],"operationName":"HTTP POST","processID":"p7","references":[{"refType":"CHILD_OF","spanID":"96f2298052cc3fda","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"7b92ebafc9a2a0f1","startTime":1750044449713664,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"network.protocol.version","type":"string","value":"1.1"},{"key":"error","type":"string","value":"unset"},{"key":"server.address","type":"string","value":"shipping"},{"key":"server.port","type":"string","value":"50050"},{"key":"url.full","type":"string","value":"http://shipping:50050/get-quote"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":391,"logs":[],"operationName":"HTTP POST","processID":"p7","references":[{"refType":"CHILD_OF","spanID":"7683762fa74ffd1c","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"92345ad5d7cb4190","startTime":1750044449746559,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"network.protocol.version","type":"string","value":"1.1"},{"key":"error","type":"string","value":"unset"},{"key":"server.address","type":"string","value":"shipping"},{"key":"server.port","type":"string","value":"50050"},{"key":"url.full","type":"string","value":"http://shipping:50050/ship-order"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":4711,"logs":[],"operationName":"POST","processID":"p9","references":[{"refType":"CHILD_OF","spanID":"64e503f233846241","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"31d9931c1b054f86","startTime":1750044449749545,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"System.Net.Http"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"network.protocol.version","type":"string","value":"2"},{"key":"server.address","type":"string","value":"flagd"},{"key":"server.port","type":"string","value":"8013"},{"key":"url.full","type":"string","value":"http://flagd:8013/flagd.evaluation.v1.Service/ResolveBoolean"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":15663,"logs":[],"operationName":"HTTP POST","processID":"p7","references":[{"refType":"CHILD_OF","spanID":"7683762fa74ffd1c","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"d96adf1246ad7d75","startTime":1750044449759771,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"network.protocol.version","type":"string","value":"1.1"},{"key":"error","type":"string","value":"unset"},{"key":"server.address","type":"string","value":"email"},{"key":"server.port","type":"string","value":"6060"},{"key":"url.full","type":"string","value":"http://email:6060/send_order_confirmation"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":3076,"logs":[],"operationName":"grpc.oteldemo.PaymentService/Charge","processID":"p4","references":[{"refType":"CHILD_OF","spanID":"530667cc212dd6ed","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"df89f1712cb9fdec","startTime":1750044449742000,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"@opentelemetry/instrumentation-grpc"},{"key":"otel.scope.version","type":"string","value":"0.57.1"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"Charge"},{"key":"rpc.service","type":"string","value":"oteldemo.PaymentService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"error","type":"string","value":"unset"},{"key":"app.payment.amount","type":"string","value":"1102.50"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":79737,"logs":[],"operationName":"POST","processID":"p10","references":[],"spanID":"10d27d153c44c541","startTime":1750044449700847,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"opentelemetry.instrumentation.requests"},{"key":"otel.scope.version","type":"string","value":"0.55b0"},{"key":"http.method","type":"string","value":"POST"},{"key":"http.status_code","type":"string","value":"200"},{"key":"http.url","type":"string","value":"http://frontend-proxy:8080/api/checkout"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":421,"logs":[{"timestamp":1750044449755249,"fields":[{"key":"event","type":"string","value":"Enqueued"}]},{"timestamp":1750044449755262,"fields":[{"key":"event","type":"string","value":"Sent"}]},{"timestamp":1750044449755655,"fields":[{"key":"event","type":"string","value":"ResponseReceived"}]}],"operationName":"HMSET","processID":"p9","references":[{"refType":"CHILD_OF","spanID":"d8802687844ff0da","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"5f78a21a81d1a9a3","startTime":1750044449755233,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"OpenTelemetry.Instrumentation.StackExchangeRedis"},{"key":"otel.scope.version","type":"string","value":"1.11.0-beta.2"},{"key":"db.redis.database_index","type":"string","value":"0"},{"key":"db.redis.flags","type":"string","value":"DemandMaster"},{"key":"db.system","type":"string","value":"redis"},{"key":"server.address","type":"string","value":"valkey-cart"},{"key":"server.port","type":"string","value":"6379"},{"key":"error","type":"string","value":"unset"},{"key":"db.statement","type":"string","value":"HMSET d526648e-4a61-11f0-8b6b-b20e5443dfb5"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":5855,"logs":[],"operationName":"flagd.evaluation.v1.Service/ResolveBoolean","processID":"p9","references":[{"refType":"CHILD_OF","spanID":"d8802687844ff0da","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"64e503f233846241","startTime":1750044449749012,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"OpenTelemetry.Instrumentation.GrpcNetClient"},{"key":"otel.scope.version","type":"string","value":"1.11.0-beta.2"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"ResolveBoolean"},{"key":"rpc.service","type":"string","value":"flagd.evaluation.v1.Service"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"server.address","type":"string","value":"flagd"},{"key":"server.port","type":"string","value":"8013"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":136,"logs":[{"timestamp":1750044449752991,"fields":[{"key":"message.id","type":"string","value":"1"},{"key":"message.type","type":"string","value":"RECEIVED"},{"key":"event","type":"string","value":"message"},{"key":"message.uncompressed_size","type":"string","value":"15"}]},{"timestamp":1750044449753111,"fields":[{"key":"message.id","type":"string","value":"1"},{"key":"message.type","type":"string","value":"SENT"},{"key":"message.uncompressed_size","type":"string","value":"15"},{"key":"event","type":"string","value":"message"}]}],"operationName":"flagd.evaluation.v1.Service/ResolveBoolean","processID":"p5","references":[{"refType":"CHILD_OF","spanID":"31d9931c1b054f86","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"9d054ff4aeb2b518","startTime":1750044449752984,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"connectrpc.com/otelconnect"},{"key":"otel.scope.version","type":"string","value":"semver:0.6.0-dev"},{"key":"rpc.method","type":"string","value":"ResolveBoolean"},{"key":"rpc.service","type":"string","value":"flagd.evaluation.v1.Service"},{"key":"error","type":"string","value":"unset"},{"key":"net.peer.name","type":"string","value":"172.18.0.10"},{"key":"net.peer.port","type":"string","value":"46838"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.system","type":"string","value":"grpc"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":877,"logs":[{"timestamp":1750044449755696,"fields":[{"key":"event","type":"string","value":"Enqueued"}]},{"timestamp":1750044449755708,"fields":[{"key":"event","type":"string","value":"Sent"}]},{"timestamp":1750044449756563,"fields":[{"key":"event","type":"string","value":"ResponseReceived"}]}],"operationName":"EXPIRE","processID":"p9","references":[{"refType":"CHILD_OF","spanID":"d8802687844ff0da","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"4a42b7a5fa81bdfb","startTime":1750044449755686,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"OpenTelemetry.Instrumentation.StackExchangeRedis"},{"key":"otel.scope.version","type":"string","value":"1.11.0-beta.2"},{"key":"db.redis.database_index","type":"string","value":"0"},{"key":"db.redis.flags","type":"string","value":"DemandMaster"},{"key":"db.system","type":"string","value":"redis"},{"key":"server.address","type":"string","value":"valkey-cart"},{"key":"server.port","type":"string","value":"6379"},{"key":"error","type":"string","value":"unset"},{"key":"db.statement","type":"string","value":"EXPIRE d526648e-4a61-11f0-8b6b-b20e5443dfb5"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":2157,"logs":[],"operationName":"oteldemo.CurrencyService/Convert","processID":"p7","references":[{"refType":"CHILD_OF","spanID":"96f2298052cc3fda","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"34a9d7aa3afe1688","startTime":1750044449711310,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"Convert"},{"key":"rpc.service","type":"string","value":"oteldemo.CurrencyService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"server.address","type":"string","value":"172.18.0.18"},{"key":"server.port","type":"string","value":"7001"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":2021,"logs":[],"operationName":"oteldemo.CurrencyService/Convert","processID":"p7","references":[{"refType":"CHILD_OF","spanID":"96f2298052cc3fda","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"11295d69d0e661dd","startTime":1750044449735781,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"Convert"},{"key":"rpc.service","type":"string","value":"oteldemo.CurrencyService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"server.address","type":"string","value":"172.18.0.18"},{"key":"server.port","type":"string","value":"7001"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":77796,"logs":[],"operationName":"POST /api/checkout","processID":"p3","references":[{"refType":"CHILD_OF","spanID":"47c48aa63a0c5a3d","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"01468af9419620f5","startTime":1750044449701000,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"next.js"},{"key":"otel.scope.version","type":"string","value":"0.0.1"},{"key":"http.method","type":"string","value":"POST"},{"key":"http.status_code","type":"string","value":"200"},{"key":"http.target","type":"string","value":"/api/checkout"},{"key":"next.rsc","type":"string","value":"false"},{"key":"next.span_name","type":"string","value":"POST /api/checkout"},{"key":"next.span_type","type":"string","value":"BaseServer.handleRequest"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":19397,"logs":[],"operationName":"POST quote","processID":"p6","references":[{"refType":"CHILD_OF","spanID":"599cbbf8e81ddaca","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"09b03b9b5481c29c","startTime":1750044449715774,"tags":[{"key":"span.kind","type":"string","value":"client"},{"key":"otel.scope.name","type":"string","value":"opentelemetry-instrumentation-actix-web"},{"key":"otel.scope.version","type":"string","value":"0.22.0"},{"key":"http.request.method","type":"string","value":"POST"},{"key":"http.response.status_code","type":"string","value":"200"},{"key":"server.address","type":"string","value":"quote"},{"key":"server.port","type":"string","value":"8090"},{"key":"url.full","type":"string","value":"http://quote:8090/getquote"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":75,"logs":[{"timestamp":1750044449711020,"fields":[{"key":"event","type":"string","value":"Product Found"}]}],"operationName":"oteldemo.ProductCatalogService/GetProduct","processID":"p11","references":[{"refType":"CHILD_OF","spanID":"7e5e7c2f1ea9cb0b","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"5b997902f830009b","startTime":1750044449710969,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"GetProduct"},{"key":"rpc.service","type":"string","value":"oteldemo.ProductCatalogService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"error","type":"string","value":"unset"},{"key":"app.product.id","type":"string","value":"0PUK6V6EV0"},{"key":"app.product.name","type":"string","value":"Solar System Color Imager"},{"key":"server.address","type":"string","value":"172.18.0.23"},{"key":"server.port","type":"string","value":"56058"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":78,"logs":[{"timestamp":1750044449778775,"fields":[{"key":"event","type":"string","value":"Product Found"}]}],"operationName":"oteldemo.ProductCatalogService/GetProduct","processID":"p11","references":[{"refType":"CHILD_OF","spanID":"394722a3d65e5bee","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"212f00429ff724f5","startTime":1750044449778734,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"},{"key":"otel.scope.version","type":"string","value":"0.61.0"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"GetProduct"},{"key":"rpc.service","type":"string","value":"oteldemo.ProductCatalogService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"error","type":"string","value":"unset"},{"key":"app.product.id","type":"string","value":"0PUK6V6EV0"},{"key":"app.product.name","type":"string","value":"Solar System Color Imager"},{"key":"server.address","type":"string","value":"172.18.0.24"},{"key":"server.port","type":"string","value":"47538"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":597,"logs":[{"timestamp":1750044449711719,"fields":[{"key":"event","type":"string","value":"Processing currency conversion request"}]},{"timestamp":1750044449711741,"fields":[{"key":"event","type":"string","value":"Conversion successful, response sent back"}]}],"operationName":"Currency/Convert","processID":"p12","references":[{"refType":"CHILD_OF","spanID":"34a9d7aa3afe1688","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"42e4324fcb045b99","startTime":1750044449711715,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"currency"},{"key":"app.currency.conversion.from","type":"string","value":"USD"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"Convert"},{"key":"rpc.service","type":"string","value":"oteldemo.CurrencyService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"error","type":"string","value":"false"},{"key":"app.currency.conversion.to","type":"string","value":"USD"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":655,"logs":[{"timestamp":1750044449736390,"fields":[{"key":"event","type":"string","value":"Processing currency conversion request"}]},{"timestamp":1750044449736414,"fields":[{"key":"event","type":"string","value":"Conversion successful, response sent back"}]}],"operationName":"Currency/Convert","processID":"p12","references":[{"refType":"CHILD_OF","spanID":"11295d69d0e661dd","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"adb556f3c99b633d","startTime":1750044449736386,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"otel.scope.name","type":"string","value":"currency"},{"key":"app.currency.conversion.from","type":"string","value":"USD"},{"key":"rpc.grpc.status_code","type":"string","value":"0"},{"key":"rpc.method","type":"string","value":"Convert"},{"key":"rpc.service","type":"string","value":"oteldemo.CurrencyService"},{"key":"rpc.system","type":"string","value":"grpc"},{"key":"error","type":"string","value":"false"},{"key":"app.currency.conversion.to","type":"string","value":"USD"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null},{"duration":78648,"logs":[],"operationName":"ingress","processID":"p8","references":[{"refType":"CHILD_OF","spanID":"10d27d153c44c541","traceID":"9e06226196051d9c3c10dfab343791ad"}],"spanID":"d66da216bedd159f","startTime":1750044449701298,"tags":[{"key":"span.kind","type":"string","value":"server"},{"key":"component","type":"string","value":"proxy"},{"key":"downstream_cluster","type":"string","value":"-"},{"key":"http.protocol","type":"string","value":"HTTP/1.1"},{"key":"node_id","type":"string","value":"-"},{"key":"peer.address","type":"string","value":"172.18.0.25"},{"key":"zone","type":"string","value":"-"},{"key":"guid:x-request-id","type":"string","value":"347edd6d-e273-953e-87f6-7ba07f352331"},{"key":"http.method","type":"string","value":"POST"},{"key":"http.status_code","type":"string","value":"200"},{"key":"http.url","type":"string","value":"http://frontend-proxy:8080/api/checkout"},{"key":"request_size","type":"string","value":"388"},{"key":"response_flags","type":"string","value":"-"},{"key":"response_size","type":"string","value":"857"},{"key":"upstream_cluster","type":"string","value":"frontend"},{"key":"upstream_cluster.name","type":"string","value":"frontend"},{"key":"user_agent","type":"string","value":"python-requests/2.32.4"},{"key":"error","type":"string","value":"unset"}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null}],"traceID":"9e06226196051d9c3c10dfab343791ad","warnings":null}],"errors":null,"limit":0,"offset":0,"total":1}
    

Previous Visualization in Jaeger UI