Skip to content

Commit

Permalink
add kafka reporter
Browse files Browse the repository at this point in the history
  • Loading branch information
lu-xiaoshuang authored and luxiaoshuang committed Jun 25, 2024
1 parent ed1d377 commit c73095f
Show file tree
Hide file tree
Showing 54 changed files with 2,193 additions and 536 deletions.
24 changes: 21 additions & 3 deletions CONTIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,32 @@ git clone https://github.com/SkyAPM/SkyAPM-dotnet.git
git submodule update --init
```

If this process is hung or has problems, please download all the contents from https://github.com/apache/skywalking-data-collect-protocol manually and put them in src/SkyApm.Transport.Protocol/protocol-v3 directory.

3. build

* Debug Mode

```
// Restore dependencies
dotnet restore
// Build Protocol
dotnet build src/SkyApm.Transport.Protocol --no-restore -c debug
// Build
dotnet build --no-restore -c debug
// Test
dotnet test --no-build --verbosity normal --framework net6.0 -c debug
```

* Release Mode

```
// Restore dependencies
dotnet restore
// Build Protocol
dotnet build src/SkyApm.Transport.Protocol --no-restore
dotnet build src/SkyApm.Transport.Protocol --no-restore -c release
// Build
dotnet build --no-restore
dotnet build --no-restore -c release
// Test
dotnet test --no-build --verbosity normal --framework net6.0
dotnet test --no-build --verbosity normal --framework net6.0 -c release
```
26 changes: 18 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@ SkyAPM C#/.NET instrument agent

## Nuget Packages

| Package Name | NuGet | MyGet | Downloads
|--------------| ------- | ------- | ----
| SkyAPM.Agent.AspNetCore | [![nuget](https://img.shields.io/nuget/v/SkyAPM.Agent.AspNetCore.svg?style=flat-square)](https://www.nuget.org/packages/SkyAPM.Agent.AspNetCore) | [![myget](https://img.shields.io/myget/skyapm-dotnet/vpre/SkyAPM.Agent.AspNetCore.svg?style=flat-square)](https://www.myget.org/feed/skyapm-dotnet/package/nuget/SkyAPM.Agent.AspNetCore) | [![stats](https://img.shields.io/nuget/dt/SkyAPM.Agent.AspNetCore.svg?style=flat-square)](https://www.nuget.org/stats/packages/SkyAPM.Agent.AspNetCore?groupby=Version)
| SkyAPM.Agent.GeneralHost | [![nuget](https://img.shields.io/nuget/v/SkyAPM.Agent.GeneralHost.svg?style=flat-square)](https://www.nuget.org/packages/SkyAPM.Agent.GeneralHost) | [![myget](https://img.shields.io/myget/skyapm-dotnet/vpre/SkyAPM.Agent.GeneralHost.svg?style=flat-square)](https://www.myget.org/feed/skyapm-dotnet/package/nuget/SkyAPM.Agent.GeneralHost) | [![](https://img.shields.io/nuget/dt/SkyAPM.Agent.GeneralHost.svg?style=flat-square)](https://www.nuget.org/stats/packages/SkyAPM.Agent.GeneralHost?groupby=Version)
| Package Name | NuGet | MyGet | Downloads
|--------------| ------- | ------- | ----
| SkyAPM.Agent.AspNetCore | [![nuget](https://img.shields.io/nuget/v/SkyAPM.Agent.AspNetCore.svg?style=flat-square)](https://www.nuget.org/packages/SkyAPM.Agent.AspNetCore) | [![myget](https://img.shields.io/myget/skyapm-dotnet/vpre/SkyAPM.Agent.AspNetCore.svg?style=flat-square)](https://www.myget.org/feed/skyapm-dotnet/package/nuget/SkyAPM.Agent.AspNetCore) | [![stats](https://img.shields.io/nuget/dt/SkyAPM.Agent.AspNetCore.svg?style=flat-square)](https://www.nuget.org/stats/packages/SkyAPM.Agent.AspNetCore?groupby=Version)
| SkyAPM.Agent.GeneralHost | [![nuget](https://img.shields.io/nuget/v/SkyAPM.Agent.GeneralHost.svg?style=flat-square)](https://www.nuget.org/packages/SkyAPM.Agent.GeneralHost) | [![myget](https://img.shields.io/myget/skyapm-dotnet/vpre/SkyAPM.Agent.GeneralHost.svg?style=flat-square)](https://www.myget.org/feed/skyapm-dotnet/package/nuget/SkyAPM.Agent.GeneralHost) | [![](https://img.shields.io/nuget/dt/SkyAPM.Agent.GeneralHost.svg?style=flat-square)](https://www.nuget.org/stats/packages/SkyAPM.Agent.GeneralHost?groupby=Version)

> MyGet feed URL https://www.myget.org/F/skyapm-dotnet/api/v3/index.json
Expand All @@ -43,7 +43,7 @@ A quick list of SkyWalking .NET Core Agent's capabilities
## Deploy SkyWalking Backend And UI

#### Requirements
Start with v1.0.0, SkyAPM .NET Core Agent only supports SkyWalking 8.0 or higher. The SkyWalking doc is [here](https://skywalking.apache.org/docs/).
Start with v1.0.0, SkyAPM .NET Core Agent only supports SkyWalking 8.0 or higher. The SkyWalking doc is [here](https://skywalking.apache.org/docs/).

## Install SkyWalking .NET Core Agent

Expand All @@ -54,7 +54,7 @@ dotnet add package SkyAPM.Agent.AspNetCore
```

## How to use
Set the `ASPNETCORE_HOSTINGSTARTUPASSEMBLIES` environment variable to support the activation of the SkyAPM .NET Core Agent.
Set the `ASPNETCORE_HOSTINGSTARTUPASSEMBLIES` environment variable to support the activation of the SkyAPM .NET Core Agent.

- Add the assembly name of `SkyAPM.Agent.AspNetCore` to the `ASPNETCORE_HOSTINGSTARTUPASSEMBLIES` environment variable.

Expand Down Expand Up @@ -89,20 +89,30 @@ Install `SkyAPM.DotNet.CLI`
dotnet tool install -g SkyAPM.DotNet.CLI
```

Use `dotnet skyapm config [your_service_name] [your_servers]` to generate config file.
For 2.2.x or older, use `dotnet skyapm config [your_service_name] [your_servers]` to generate config file.

```
dotnet skyapm config sample_app 192.168.0.1:11800
```

For 2.3.x or newer, use `dotnet skyapm config ${your_service_name} [--reporter=grpc|kafka] [--grpcservers=...] [--kafkaservers=...]` to generate config file.

```
# grpc
dotnet skyapm config sample_app --reporter=grpc --grpcservers=192.168.0.1:11800
# kafka
dotnet skyapm config sample_app --reporter=kafka --kafkaservers=192.168.0.1:9092
```

# Contributing
This section is in progress here: [Contributing to SkyAPM-dotnet](/CONTIBUTING.md)

# Contact Us
* Submit an issue

If you have issues about SkyWalking protocol, its official backend, ask questions at their Apache official channels. All following channels are not suitable for .net agent, but good if you are facing backend/UI issues.
* Submit an official Apache SkyWalking [issue](https://github.com/apache/skywalking/issues).
* Submit an official Apache SkyWalking [issue](https://github.com/apache/skywalking/issues).
* Mail list: **[email protected]**. Mail to `[email protected]`, follow the reply to subscribe the mail list.
* Join `skywalking` channel at [Apache Slack](https://join.slack.com/t/the-asf/shared_invite/enQtNzc2ODE3MjI1MDk1LTAyZGJmNTg1NWZhNmVmOWZjMjA2MGUyOGY4MjE5ZGUwOTQxY2Q3MDBmNTM5YTllNGU4M2QyMzQ4M2U4ZjQ5YmY). If the link is not working, find the latest one at [Apache INFRA WIKI](https://cwiki.apache.org/confluence/display/INFRA/Slack+Guest+Invites).
* QQ Group: 392443393(2000/2000, not available), 901167865(available)
Expand Down
12 changes: 6 additions & 6 deletions build/sign.props
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
<Project>
<PropertyGroup Condition="'$(Configuration)' == 'Release'">
<SignAssembly>True</SignAssembly>
<AssemblyOriginatorKeyFile>$(MSBuildThisFileDirectory)SkyAPM.snk</AssemblyOriginatorKeyFile>
<DelaySign>False</DelaySign>
</PropertyGroup>
</Project>
<PropertyGroup Condition="'$(Configuration)' == 'Release'">
<SignAssembly>True</SignAssembly>
<AssemblyOriginatorKeyFile>$(MSBuildThisFileDirectory)SkyAPM.snk</AssemblyOriginatorKeyFile>
<DelaySign>False</DelaySign>
</PropertyGroup>
</Project>
14 changes: 7 additions & 7 deletions build/version.props
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
<Project>
<PropertyGroup>
<VersionMajor>2</VersionMajor>
<VersionMinor>2</VersionMinor>
<VersionPatch>0</VersionPatch>
<VersionQuality></VersionQuality>
<VersionPrefix>$(VersionMajor).$(VersionMinor).$(VersionPatch)</VersionPrefix>
</PropertyGroup>
<PropertyGroup>
<VersionMajor>2</VersionMajor>
<VersionMinor>2</VersionMinor>
<VersionPatch>0</VersionPatch>
<VersionQuality></VersionQuality>
<VersionPrefix>$(VersionMajor).$(VersionMinor).$(VersionPatch)</VersionPrefix>
</PropertyGroup>
</Project>
47 changes: 47 additions & 0 deletions sample/grpc/SkyApm.Sample.GrpcClient/Extensions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
/*
* Licensed to the SkyAPM under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The SkyAPM licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/

using System;
using System.Threading;

using Grpc.Net.Client;
using GrpcGreeter;
using Microsoft.Extensions.DependencyInjection;
using SkyApm.Diagnostics.Grpc.Client;

namespace SkyApm.Sample.GrpcClient
{
public static class Extensions
{
public static IServiceProvider StartGrpcClient(this IServiceProvider provider)
{
GrpcChannel channel = GrpcChannel.ForAddress("http://localhost:12345");
var interceptor = provider.GetService<ClientDiagnosticInterceptor>();
Greeter.GreeterClient greeterClient = new Greeter.GreeterClient(channel);
for (int i = 0; i < 1000; ++ i)
{
HelloRequest helloRequest = new HelloRequest();
helloRequest.Name = "hello " + i;
HelloReply helloReply = greeterClient.SayHello(helloRequest);
Console.WriteLine(helloReply.Message);
Thread.Sleep(10000);
}
return provider;
}
}
}
50 changes: 50 additions & 0 deletions sample/grpc/SkyApm.Sample.GrpcClient/Program.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
/*
* Licensed to the SkyAPM under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The SkyAPM licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/

using System;
using System.Threading.Tasks;

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using SkyApm.Agent.GeneralHost;

namespace SkyApm.Sample.GrpcClient
{
class Program
{
public static async Task Main(string[] args)
{
var host = CreateHostBuilder(args).Build();
host.Services.StartGrpcClient();
await host.RunAsync();
}

public static IHostBuilder CreateHostBuilder(string[] args)
{
var environmentName = Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT");

return new HostBuilder()
.UseEnvironment(environmentName)
.AddSkyAPM()
.ConfigureServices((hostContext, services) =>
{
services.AddLogging();
});
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"profiles": {
"SkyApm.Sample.GrpcClient": {
"commandName": "Project",
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
}
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<GeneratePackageOnBuild>True</GeneratePackageOnBuild>
<OutputType>Exe</OutputType>
<TargetFrameworks>netcoreapp3.1;net5.0;net6.0;net7.0</TargetFrameworks>
</PropertyGroup>

<ItemGroup>
<Content Include="skyapm.json">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
<CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
</Content>
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)' == 'netcoreapp3.1'">
<PackageReference Include="Microsoft.Extensions.Hosting" Version="3.1.32" PrivateAssets="All"/>
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)' == 'net5.0'">
<PackageReference Include="Microsoft.Extensions.Hosting" Version="5.0.0"/>
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)' == 'net6.0'">
<PackageReference Include="Microsoft.Extensions.Hosting" Version="6.0.1"/>
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)' == 'net7.0'">
<PackageReference Include="Microsoft.Extensions.Hosting" Version="7.0.0"/>
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\..\src\SkyApm.Agent.GeneralHost\SkyApm.Agent.GeneralHost.csproj"/>
<ProjectReference Include="..\SkyApm.Sample.GrpcProto\SkyApm.Sample.GrpcProto.csproj"/>
</ItemGroup>
</Project>
47 changes: 47 additions & 0 deletions sample/grpc/SkyApm.Sample.GrpcClient/skyapm.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
{
"SkyWalking": {
"ServiceName": "grpc-greeter-client",
"Namespace": "",
"HeaderVersions": [
"sw8"
],
"Sampling": {
"SamplePer3Secs": -1,
"Percentage": -1.0
},
"Logging": {
"Level": "Information",
"FilePath": "logs/skyapm-{Date}.log"
},
"MeterActive": true,
"MetricActive": true,
"SegmentActive": true,
"ProfilingActive": true,
"ManagementActive": true,
"LogActive": true,
"Transport": {
"Interval": 3000,
"ProtocolVersion": "v8",
"QueueSize": 30000,
"BatchSize": 3000,
"Reporter": "kafka",
"gRPC": {
"Servers": "localhost:11800",
"Timeout": 100000,
"ConnectTimeout": 100000,
"ReportTimeout": 600000
},
"Kafka": {
"BootstrapServers": "localhost:19091,localhost:19092,localhost:19093",
"TopicTimeoutMs": 3000,
"MessageTimeoutMs": 10000,
"TopicMeters": "skywalking-meters",
"TopicCLRMetrics": "skywalking-clr-metrics",
"TopicSegments": "skywalking-segments",
"TopicProfilings": "skywalking-profilings",
"TopicManagements": "skywalking-managements",
"TopicLogs": "skywalking-logs"
}
}
}
}
26 changes: 21 additions & 5 deletions sample/grpc/SkyApm.Sample.GrpcServer/Extensions.cs
Original file line number Diff line number Diff line change
@@ -1,11 +1,27 @@
using Grpc.Core;
/*
* Licensed to the SkyAPM under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The SkyAPM licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/

using System;
using Grpc.Core;
using Grpc.Core.Interceptors;
using GrpcGreeter;
using SkyApm.Diagnostics.Grpc.Server;
using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.Extensions.DependencyInjection;
using SkyApm.Diagnostics.Grpc.Server;

namespace SkyApm.Sample.GrpcServer
{
Expand Down
22 changes: 20 additions & 2 deletions sample/grpc/SkyApm.Sample.GrpcServer/GreeterImpl.cs
Original file line number Diff line number Diff line change
@@ -1,9 +1,27 @@
using Grpc.Core;
using GrpcGreeter;
/*
* Licensed to the SkyAPM under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The SkyAPM licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/

using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Threading.Tasks;
using Grpc.Core;
using GrpcGreeter;

namespace SkyApm.Sample.GrpcServer
{
Expand Down
Loading

0 comments on commit c73095f

Please sign in to comment.