4c4ce1a23 · cryptotaxi247/netdata-1

@@ -1,268 +0,0 @@

1 -# Deployment strategies

2 -

3 -Netdata can be used to monitor all kinds of infrastructure, from stand-alone tiny IoT devices to complex hybrid setups

4 -combining on-premise and cloud infrastructure, mixing bare-metal servers, virtual machines and containers.

5 -

6 -There are 3 components to structure your Netdata ecosystem:

7 -

8 -1. **Netdata Agents**

9 - To monitor the physical or virtual nodes of your infrastructure, including all applications and containers running on them.

10 -

11 - Netdata Agents are Open-Source, licensed under GPL v3+.

12 -

13 -2. **Netdata Parents**

14 - To create data centralization points within your infrastructure, to offload Netdata Agents functions from your production

15 - systems, to provide high-availability of your data, increased data retention and isolation of your nodes.

16 -

17 - Netdata Parents are implemented using the Netdata Agent software. Any Netdata Agent can be an Agent for a node and a Parent

18 - for other Agents, at the same time.

19 -

20 - It is recommended to set up multiple Netdata Parents. They will all seamlessly be integrated by Netdata Cloud into one monitoring solution.

21 -

22 -

23 -3. **Netdata Cloud**

24 - Our SaaS, combining all your infrastructure, all your Netdata Agents and Parents, into one uniform, distributed,

25 - scalable, monitoring database, offering advanced data slicing and dicing capabilities, custom dashboards, advanced troubleshooting

26 - tools, user management, centralized management of alerts, and more.

27 -

28 -

29 -The Netdata Agent is a highly modular software piece, providing data collection via numerous plugins, an in-house crafted time-series

30 -database, a query engine, health monitoring and alerts, machine learning and anomaly detection, metrics exporting to third party systems.

31 -

32 -

33 -## Deployment Options Overview

34 -

35 -This section provides a quick overview of a few common deployment options. The next sections go into configuration examples and further reading.

36 -

37 -### Stand-alone Deployment

38 -

39 -To help our users have a complete experience of Netdata when they install it for the first time, a Netdata Agent with default configuration

40 -is a complete monitoring solution out of the box, having all these features enabled and available.

41 -

42 -The Agent will act as a _stand-alone_ Agent by default, and this is great to start out with for small setups and home labs. By [connecting each Agent to Cloud](https://github.com/netdata/netdata/blob/master/src/claim/README.md), you can see an overview of all your nodes, with aggregated charts and centralized alerting, without setting up a Parent.

43 -

44 -![image](https://github.com/netdata/netdata/assets/116741/6a638175-aec4-4d46-85a6-520c283ab6a8)

45 -

46 -### Parent – Child Deployment

47 -

48 -An Agent connected to a Parent is called a _Child_. It will _stream_ metrics to its Parent. The Parent can then take care of storing metrics on behalf of that node (with longer retention), handle metrics queries for showing dashboards, and provide alerting.

49 -

50 -When using Cloud, it is recommended that just the Parent is connected to Cloud. Child Agents can then be configured to have short retention, in RAM instead of on Disk, and have alerting and other features disabled. Because they don't need to connect to Cloud themselves, those children can then be further secured by not allowing outbound traffic.

51 -

52 -![image](https://github.com/netdata/netdata/assets/116741/cb65698d-a6b7-43ee-a2d1-c30d0a46f084)

53 -

54 -This setup allows for leaner Child nodes and is good for setups with more than a handful of nodes. Metrics data remains accessible if the Child node is temporarily unavailable or decommissioned, although there is no failover in case the Parent becomes unavailable.

55 -

56 -

57 -### Active–Active Parent Deployment

58 -

59 -For high availability, Parents can be configured to stream data for their children between them, and keep the data sets in sync. Child Agents are configured with the addresses of both Parent Agents, but will only stream to one of them at a time. When that Parent becomes unavailable, it reconnects to another. When the first Parent becomes available again, that Parent will catch up by receiving the backlog from the second.

60 -

61 -With both Parent Agents connected to Cloud, Cloud will route queries to either Parent transparently, depending on their availability. Alerts trigger on either Parent will stream to Cloud, and Cloud will deduplicate and debounce state changes to prevent spurious notifications.

62 -

63 -![image](https://github.com/netdata/netdata/assets/116741/6ae2b10c-7f7d-4503-aac4-0a9381c6f80b)

64 -

65 -

66 -## Configuration Details

67 -

68 -### Stand-alone Deployment

69 -

70 -The stand-alone setup is configured out of the box with reasonable defaults, but please consult our [configuration documentation](https://github.com/netdata/netdata/blob/master/docs/cloud/cheatsheet.md) for details, including the overview of [common configuration changes](https://github.com/netdata/netdata/blob/master/docs/configure/common-changes.md).

71 -

72 -### Parent – Child Deployment

73 -

74 -For setups involving Child and Parent Agents, the Agents need to be configured for [_streaming_](https://github.com/netdata/netdata/blob/master/src/streaming/README.md), through the configuration file `stream.conf`. This will instruct the Child to stream data to the Parent and the Parent to accept streaming connections for one or more Child Agents. To secure this connection, both need set up a shared API key (to replace the string `API_KEY` in the examples below). Additionally, the Child is configured with one or more addresses of Parent Agents (`PARENT_IP_ADDRESS`).

75 -

76 -An API key is a key created with `uuidgen` and is used for authentication and/or customization in the Parent side. I.e. a Child will stream using the API key, and a Parent is configured to accept connections from Child, but can also apply different options for children by using multiple different API keys. The easiest setup uses just one API key for all Child Agents.

77 -

78 -#### Child config

79 -

80 -As mentioned above, the recommendation is to not claim the Child to Cloud directly during your setup, avoiding establishing an [ACLK](https://github.com/netdata/netdata/blob/master/src/aclk/README.md) connection.

81 -

82 -To reduce the footprint of the Netdata Agent on your production system, some capabilities can be switched OFF on the Child and kept ON on the Parent. In this example, Machine Learning and Alerting are disabled in the Child, so that the Parent can take the load. We also use RAM instead of disk to store metrics with limited retention, covering temporary network issues.

83 -

84 -##### netdata.conf

85 -

86 -On the child node, edit `netdata.conf` by using the edit-config script: `/etc/netdata/edit-config netdata.conf` set the following parameters:

87 -

88 -```yaml

89 -[db]

90 - # https://learn.netdata.cloud/docs/agent/database

91 - # none = no retention, ram = some retention in ram

92 - mode = ram

93 - # The retention in seconds.

94 - # This provides some tolerance to the time the child has to find a parent in

95 - # order to transfer the data. For IoT this can be lowered to 120.

96 - retention = 1200

97 - # The granularity of metrics, in seconds.

98 - # You may increase this to lower CPU resources.

99 - update every = 1

100 -[ml]

101 - # Disable Machine Learning

102 - enabled = no

103 -[health]

104 - # Disable Health Checks (Alerting)

105 - enabled = no

106 -[web]

107 - # Disable remote access to the local dashboard

108 - bind to = lo

109 -[plugins]

110 - # Uncomment the following line to disable all external plugins on extreme

111 - # IoT cases by default.

112 - # enable running new plugins = no

113 -```

114 -

115 -##### stream.conf

116 -

117 -To edit `stream.conf`, again use the edit-config script: `/etc/netdata/edit-config stream.conf`.

118 -

119 -Set the following parameters:

120 -

121 -```yaml

122 -[stream]

123 - # Stream metrics to another Netdata

124 - enabled = yes

125 - # The IP and PORT of the parent

126 - destination = PARENT_IP_ADDRESS:19999

127 - # The shared API key, generated by uuidgen

128 - api key = API_KEY

129 -```

130 -

131 -#### Parent config

132 -

133 -For the Parent, besides setting up streaming, the example will also provide an example configuration of multiple [tiers](https://github.com/netdata/netdata/blob/master/src/database/engine/README.md#tiering) of metrics [storage](https://github.com/netdata/netdata/blob/master/docs/store/change-metrics-storage.md), for 10 children, with about 2k metrics each.

134 -

135 -- 1s granularity at tier 0 for 1 week

136 -- 1m granularity at tier 1 for 1 month

137 -- 1h granularity at tier 2 for 1 year

138 -

139 -Requiring:

140 -

141 -- 25GB of disk

142 -- 3.5GB of RAM (2.5GB under pressure)

143 -

144 -##### netdata.conf

145 -

146 -On the Parent, edit `netdata.conf` with `/etc/netdata/edit-config netdata.conf` and set the following parameters:

147 -

148 -```yaml

149 -[db]

150 - mode = dbengine

151 - storage tiers = 3

152 - # To allow memory pressure to offload index from ram

153 - dbengine page descriptors in file mapped memory = yes

154 - # storage tier 0

155 - update every = 1

156 - dbengine multihost disk space MB = 12000

157 - dbengine page cache size MB = 1400

158 - # storage tier 1

159 - dbengine tier 1 page cache size MB = 512

160 - dbengine tier 1 multihost disk space MB = 4096

161 - dbengine tier 1 update every iterations = 60

162 - dbengine tier 1 backfill = new

163 - # storage tier 2

164 - dbengine tier 2 page cache size MB = 128

165 - dbengine tier 2 multihost disk space MB = 2048

166 - dbengine tier 2 update every iterations = 60

167 - dbengine tier 2 backfill = new

168 -[ml]

169 - # Enabled by default

170 - # enabled = yes

171 -[health]

172 - # Enabled by default

173 - # enabled = yes

174 -[web]

175 - # Enabled by default

176 - # bind to = *

177 -```

178 -

179 -##### stream.conf

180 -

181 -On the Parent node, edit `stream.conf` with `/etc/netdata/edit-config stream.conf`, and then set the following parameters:

182 -

183 -```yaml

184 -[API_KEY]

185 - # Accept metrics streaming from other Agents with the specified API key

186 - enabled = yes

187 -```

188 -

189 -### Active–Active Parent Deployment

190 -

191 -In order to setup active–active streaming between Parent 1 and Parent 2, Parent 1 needs to be instructed to stream data to Parent 2 and Parent 2 to stream data to Parent 1. The Child Agents need to be configured with the addresses of both Parent Agents. The Agent will only connect to one Parent at a time, falling back to the next if the previous failed. These examples use the same API key between Parent Agents as for connections from Child Agents.

192 -

193 -On both Netdata Parent and all Child Agents, edit `stream.conf` with `/etc/netdata/edit-config stream.conf`:

194 -

195 -##### stream.conf on Parent 1

196 -

197 -```yaml

198 -[stream]

199 - # Stream metrics to another Netdata

200 - enabled = yes

201 - # The IP and PORT of Parent 2

202 - destination = PARENT_2_IP_ADDRESS:19999

203 - # This is the API key for the outgoing connection to Parent 2

204 - api key = API_KEY

205 -[API_KEY]

206 - # Accept metrics streams from Parent 2 and Child Agents

207 - enabled = yes

208 -```

209 -

210 -##### stream.conf on Parent 2

211 -

212 -```yaml

213 -[stream]

214 - # Stream metrics to another Netdata

215 - enabled = yes

216 - # The IP and PORT of Parent 1

217 - destination = PARENT_1_IP_ADDRESS:19999

218 - api key = API_KEY

219 -[API_KEY]

220 - # Accept metrics streams from Parent 1 and Child Agents

221 - enabled = yes

222 -```

223 -

224 -##### stream.conf on Child Agents

225 -

226 -```yaml

227 -[stream]

228 - # Stream metrics to another Netdata

229 - enabled = yes

230 - # The IP and PORT of the parent

231 - destination = PARENT_1_IP_ADDRESS:19999 PARENT_2_IP_ADDRESS:19999

232 - # The shared API key, generated by uuidgen

233 - api key = API_KEY

234 -```

235 -

236 -## Further Reading

237 -

238 -We strongly recommend the following configuration changes for production deployments:

239 -

240 -1. Understand Netdata's [security and privacy design](https://github.com/netdata/netdata/blob/master/docs/security-and-privacy-design/README.md) and

241 - [secure your nodes](https://github.com/netdata/netdata/blob/master/docs/category-overview-pages/secure-nodes.md)

242 -

243 - To safeguard your infrastructure and comply with your organization's security policies.

244 -

245 -2. Set up [streaming and replication](https://github.com/netdata/netdata/blob/master/src/streaming/README.md) to:

246 -

247 - - Offload Netdata Agents running on production systems and free system resources for the production applications running on them.

248 - - Isolate production systems from the rest of the world and improve security.

249 - - Increase data retention.

250 - - Make your data highly available.

251 -

252 -3. [Optimize the Netdata Agents system utilization and performance](https://github.com/netdata/netdata/blob/master/docs/guides/configure/performance.md)

253 -

254 - To save valuable system resources, especially when running on weak IoT devices.

255 -

256 -We also suggest that you:

257 -

258 -1. [Use Netdata Cloud to access the dashboards](https://github.com/netdata/netdata/blob/master/docs/quickstart/infrastructure.md)

259 -

260 - For increased security, user management and access to our latest tools for advanced dashboarding and troubleshooting.

261 -

262 -2. [Change how long Netdata stores metrics](https://github.com/netdata/netdata/blob/master/docs/store/change-metrics-storage.md)

263 -

264 - To control Netdata's memory use, when you have a lot of ephemeral metrics.

265 -

266 -3. [Use host labels](https://github.com/netdata/netdata/blob/master/docs/guides/using-host-labels.md)

267 -

268 - To organize systems, metrics, and alerts.

Remove deployment-strategies (#17096)