Documentation: Explain --config.expand-env=true double slash with slash substitution (#6139)

* docs: Promtail configuration explain expand-env slash pitfall

* docs: link to correct envsubst go package

Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>

Co-authored-by: Kaviraj Kanagaraj <kavirajkanagaraj@gmail.com>

docs/sources/clients/promtail/configuration.md

@@ -60,6 +60,11 @@ ${VAR:default_value}
 
 Where default_value is the value to use if the environment variable is undefined.
 
+**Note**: With `expand-env=true` the configuration will first run through
+[envsubst](https://pkg.go.dev/github.com/drone/envsubst) which will replace double
+slashes with single slashes. Because of this every use of a slash `\` needs to
+be replaced with a double slash `\\`
+
 ### Generic placeholders:
 
 - `<boolean>`: a boolean that can take the values `true` or `false`

docs/sources/clients/promtail/stages/regex.md

@@ -38,6 +38,16 @@ But these are not:
 - `expression: '\\w*'` (only escape backslashes when using double quotes)
 - `expression: "\w*"` (backslash must be escaped)
 
+
+If you run Promtail with the `--config.expand-env=true` flag the configuration
+will run through [envsubst](https://pkg.go.dev/github.com/drone/envsubst)  which will
+replace double slashes with single slashes. Because of this when using
+`expand-env=true` you need to use double slashes for each single slash. For
+example:
+
+- `expression: '\w*'` must be `expression: '\\w*'`
+- `expression: "\\w*"` must be `expression: "\\\\w*"`
+
 ## Example
 
 ### Without `source`