Hello,
Sorry I am making so many topics. But each time, I am progressing and I think it deserves a new explanation. Let me know if you think this is not the right approach.
This is a follow up to https://framacolibri.org/t/help-review-docker-compose-yaml-and-process which is itself a follow up to https://framacolibri.org/t/issue-with-fresh-docker-install-invalid-client
Getting some help via Mastodon, I have found out that my docker network was set up wrong.
Each container had its own default made network via portainer, and none was connected to nginx’s. Which had me expose all the container hosts.
So what I did is:
- Deleted all network but nginx’s
- Added all containers to nginx’s network
So now my NPM looks like this:
So the domains are using actually the ports from the container, not the host. And those are not exposed to the internet. which is nice!
So. I am 99% convinced my previous wrong set up had some impact / influence on the behavior of my two previous posts… Hence I am trying again. Now I face new different issues.
Here are my steps:
.env:
# Database / Postgres service configuration
POSTGRES_USER=somesafeusername
POSTGRES_PASSWORD=somesafepassword
# Postgres database name "peertube"
POSTGRES_DB=peertube
# The database name used by PeerTube will be PEERTUBE_DB_NAME (only if set) *OR* 'peertube'+PEERTUBE_DB_SUFFIX
PEERTUBE_DB_NAME=peertube
#PEERTUBE_DB_SUFFIX=_prod
# Database username and password used by PeerTube must match Postgres', so they are copied:
PEERTUBE_DB_USERNAME=$POSTGRES_USER
PEERTUBE_DB_PASSWORD=$POSTGRES_PASSWORD
PEERTUBE_DB_SSL=false
# Default to Postgres service name "postgres" in docker-compose.yml
PEERTUBE_DB_HOSTNAME=postgres
# PeerTube server configuration
# If you test PeerTube in local: use "peertube.localhost" and add this domain to your host file resolving on 127.0.0.1
PEERTUBE_WEBSERVER_HOSTNAME=domain_or_ip_have_the_same_result
# If you just want to test PeerTube on local
PEERTUBE_WEBSERVER_PORT=9000
PEERTUBE_WEBSERVER_HTTPS=true
# If you need more than one IP as trust_proxy
# pass them as a comma separated array:
PEERTUBE_TRUST_PROXY=["127.0.0.1", "loopback", "172.26.0.0/16"]
# Generate one using `openssl rand -hex 32`
PEERTUBE_SECRET=somesecret
# E-mail configuration
# If you use a Custom SMTP server
PEERTUBE_SMTP_USERNAME=email
PEERTUBE_SMTP_PASSWORD=password
# Default to Postfix service name "postfix" in docker-compose.yml
# May be the hostname of your Custom SMTP server
PEERTUBE_SMTP_HOSTNAME=smtp
PEERTUBE_SMTP_PORT=port
PEERTUBE_SMTP_FROM=email
PEERTUBE_SMTP_TLS=false
PEERTUBE_SMTP_DISABLE_STARTTLS=false
PEERTUBE_ADMIN_EMAIL=email
# Postfix service configuration
POSTFIX_myhostname=domain
# If you need to generate a list of sub/DOMAIN keys
# pass them as a whitespace separated string <DOMAIN>=<selector>
OPENDKIM_DOMAINS=domain=peertube
# see https://github.com/wader/postfix-relay/pull/18
OPENDKIM_RequireSafeKeys=no
PEERTUBE_OBJECT_STORAGE_UPLOAD_ACL_PUBLIC="public-read"
PEERTUBE_OBJECT_STORAGE_UPLOAD_ACL_PRIVATE="private"
#PEERTUBE_LOG_LEVEL=info
# /!\ Prefer to use the PeerTube admin interface to set the following configurations /!\
#PEERTUBE_SIGNUP_ENABLED=true
#PEERTUBE_TRANSCODING_ENABLED=true
#PEERTUBE_CONTACT_FORM_ENABLED=true
my compose.yaml:
services:
# You can comment this webserver section if you want to use another webserver/proxy or test PeerTube in local
# webserver:
# image: chocobozzz/peertube-webserver:latest
# # If you don't want to use the official image and build one from sources:
# # build:
# # context: .
# # dockerfile: ./support/docker/production/Dockerfile.nginx
# env_file:
# - .env
# ports:
# - "800:80"
# - "4433:443"
# volumes:
# - type: bind
# # Switch sources if you downloaded the whole repository
# #source: ../../nginx/peertube
# source: ./docker-volume/nginx/peertube.file
# target: /etc/nginx/conf.d/peertube.template
# - assets:/var/www/peertube/peertube-latest/client/dist:ro
# - ./docker-volume/data:/var/www/peertube/storage
# - certbot-www:/var/www/certbot
# - ./docker-volume/certbot/conf:/etc/letsencrypt
# depends_on:
# - peertube
# restart: "always"
# You can comment this certbot section if you want to use another webserver/proxy or test PeerTube in local
# certbot:
# container_name: certbot
# image: certbot/certbot
# volumes:
# - ./docker-volume/certbot/conf:/etc/letsencrypt
# - certbot-www:/var/www/certbot
# restart: unless-stopped
# entrypoint: /bin/sh -c "trap exit TERM; while :; do certbot renew --webroot -w /var/www/certbot; sleep 12h & wait $${!}; done;"
# depends_on:
# - webserver
peertube:
# If you don't want to use the official image and build one from sources:
# build:
# context: .
# dockerfile: ./support/docker/production/Dockerfile.bookworm
image: chocobozzz/peertube:production-bookworm
# Use a static IP for this container because nginx does not handle proxy host change without reload
# This container could be restarted on crash or until the postgresql database is ready for connection
networks:
default:
ipv4_address: 172.26.0.20
ipv6_address: fdab:e4b3:21a2:ef1b::42
env_file:
- .env
ports:
- "1935:1935" # Comment if you don't want to use the live feature
- "9000:9000" # Uncomment if you use another webserver/proxy or test PeerTube in local, otherwise not suitable for production
volumes:
# Remove the following line if you want to use another webserver/proxy or test PeerTube in local
# - assets:/app/client/dist
- ./docker-volume/data:/data
- ./docker-volume/config:/config
depends_on:
- postgres
- redis
- postfix
restart: "always"
postgres:
image: postgres:13-alpine
env_file:
- .env
volumes:
- ./docker-volume/db:/var/lib/postgresql/data
restart: "always"
redis:
image: redis:6-alpine
volumes:
- ./docker-volume/redis:/data
restart: "always"
postfix:
image: mwader/postfix-relay
env_file:
- .env
volumes:
- ./docker-volume/opendkim/keys:/etc/opendkim/keys
restart: "always"
networks:
default:
external: true
name: ext
enable_ipv6: false
ipam:
driver: default
config:
- subnet: 172.26.0.0/16
#- subnet: fdab:e4b3:21a2:ef1b::/64
IPv6 is disabled by the way because I cant even get IPv4 to work.
And for the record, this is my « ext » network:
If I run this as is, this is the output:
xxx@xxx:~/docker/xxx$ docker compose up
[+] Running 38/4
✔ postgres Pulled 11.5s
✔ postfix Pulled 6.1s
✔ redis Pulled 2.2s
✔ peertube Pulled 46.6s
[+] Running 3/3
✔ Container feditubo-postfix-1 Created 0.7s
✔ Container feditubo-postgres-1 Created 0.7s
✔ Container feditubo-redis-1 Created 0.7s
⠋ Container feditubo-peertube-1 Creating 0.0s
Error response from daemon: invalid config for network ec5d72916863d7f1bf29f3629417a990f1b2c7ace2e671fa874e2930c674e4df: invalid endpoint settings:
* user specified IP address is supported only when connecting to networks with user configured subnets
* user specified IP address is supported only when connecting to networks with user configured subnets
IF I comment the ip assignment like this:
peertube:
# If you don't want to use the official image and build one from sources:
# build:
# context: .
# dockerfile: ./support/docker/production/Dockerfile.bookworm
image: chocobozzz/peertube:production-bookworm
# Use a static IP for this container because nginx does not handle proxy host change without reload
# This container could be restarted on crash or until the postgresql database is ready for connection
#networks:
#default:
#ipv4_address: 172.26.0.20
#ipv6_address: fdab:e4b3:21a2:ef1b::42
env_file:
- .env
the output will be:
xxx@xxx:~/docker/xxx$ docker compose up
[+] Running 4/0
✔ Container feditubo-postfix-1 Created 0.0s
✔ Container feditubo-redis-1 Created 0.0s
✔ Container feditubo-postgres-1 Created 0.0s
✔ Container feditubo-peertube-1 Created 0.0s
Attaching to peertube-1, postfix-1, postgres-1, redis-1
redis-1 | 1:C 06 Feb 2025 11:35:09.002 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis-1 | 1:C 06 Feb 2025 11:35:09.002 # Redis version=6.2.17, bits=64, commit=00000000, modified=0, pid=1, just started
redis-1 | 1:C 06 Feb 2025 11:35:09.002 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis-1 | 1:M 06 Feb 2025 11:35:09.003 * monotonic clock: POSIX clock_gettime
redis-1 | 1:M 06 Feb 2025 11:35:09.004 * Running mode=standalone, port=6379.
redis-1 | 1:M 06 Feb 2025 11:35:09.004 # Server initialized
redis-1 | 1:M 06 Feb 2025 11:35:09.004 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis-1 | 1:M 06 Feb 2025 11:35:09.005 * Loading RDB produced by version 6.2.17
redis-1 | 1:M 06 Feb 2025 11:35:09.005 * RDB age 13 seconds
redis-1 | 1:M 06 Feb 2025 11:35:09.005 * RDB memory usage when created 0.85 Mb
redis-1 | 1:M 06 Feb 2025 11:35:09.005 # Done loading RDB, keys loaded: 50, keys expired: 28.
redis-1 | 1:M 06 Feb 2025 11:35:09.005 * DB loaded from disk: 0.000 seconds
redis-1 | 1:M 06 Feb 2025 11:35:09.005 * Ready to accept connections
postgres-1 |
postgres-1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres-1 |
postgres-1 | 2025-02-06 11:35:09.048 UTC [1] LOG: starting PostgreSQL 13.18 on x86_64-pc-linux-musl, compiled by gcc (Alpine 14.2.0) 14.2.0, 64-bit
postgres-1 | 2025-02-06 11:35:09.048 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres-1 | 2025-02-06 11:35:09.048 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres-1 | 2025-02-06 11:35:09.051 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres-1 | 2025-02-06 11:35:09.059 UTC [27] LOG: database system was shut down at 2025-02-06 11:34:56 UTC
postgres-1 | 2025-02-06 11:35:09.066 UTC [1] LOG: database system is ready to accept connections
postfix-1 | DNS records:
postfix-1 | peertube._domainkey.domain. IN TXT ( "v=DKIM1; h=sha256; k=rsa; "
postfix-1 | "p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqrkOlqHH4VNjicxjvsJBeHxCSmOOcQD9D6AOK7h6FXGgc72DJJYFYgI3MxgloPYcbeCE0YhFYpASxWjou+188Rd1S5BfcEPoVADvykJ2G03BfMgZP8tIZqPVhONgEjOELXRYs6EYp89tBi5o3riso8b6WST2KfnpolWmvQste0lvNuu9IW5ekFB2GUHTCZjIowboNjWeNistWJ"
postfix-1 | "jYPKdqZoeTPrbaY5Tg5rztMGUiz5C7VPoA/6A1NOGzHBabT5A91efd7YPJxN6ZrfwaZ54njcGfzVX0Xxp3kPtRCFRO9Lh+D8iOpofERsWK8ygz1p+O2g+jiNNztoeeccGzBHsE2QIDAQAB" ) ; ----- DKIM key peertube for domain
postfix-1 | Starting OpenDKIM: opendkim.
postfix-1 | Starting Postfix Mail Transport Agent: postfix.
postfix-1 | Skipping /etc/rsyslog.conf generating - file already exists
postfix-1 | 2025-02-06T11:35:12.087797+00:00 3866e26804cc rsyslogd: [origin software="rsyslogd" swVersion="8.1901.0" x-pid="145" x-info="https://www.rsyslog.com"] start
postfix-1 | 2025-02-06T11:35:12.091637+00:00 3866e26804cc postfix/master[144]: daemon started -- version 3.4.23, configuration /etc/postfix
peertube-1 | [domain:9000] 2025-02-06 11:35:13.028 info: Using following configuration file hierarchy: /app/config/default.yaml -> /app/support/docker/production/config/production.yaml -> /app/support/docker/production/config/custom-environment-variables.yaml.
peertube-1 | [domain:9000] 2025-02-06 11:35:13.206 error: Unable to connect to PostgreSQL database. {
peertube-1 | "err": {
peertube-1 | "stack": "SequelizeConnectionError: password authentication failed for user \"yNUnREWTp8ZAhYutGHn\"\n at Client._connectionCallback (/app/node_modules/sequelize/lib/dialects/postgres/connection-manager.js:145:24)\n at Client._handleErrorWhileConnecting (/app/node_modules/pg/lib/client.js:326:19)\n at Client._handleErrorMessage (/app/node_modules/pg/lib/client.js:346:19)\n at Connection.emit (node:events:517:28)\n at /app/node_modules/pg/lib/connection.js:116:12\n at Parser.parse (/app/node_modules/pg-protocol/dist/parser.js:36:17)\n at Socket.<anonymous> (/app/node_modules/pg-protocol/dist/index.js:11:42)\n at Socket.emit (node:events:517:28)\n at addChunk (node:internal/streams/readable:368:12)\n at readableAddChunk (node:internal/streams/readable:341:9)\n at Readable.push (node:internal/streams/readable:278:10)\n at TCP.onStreamRead (node:internal/stream_base_commons:190:23)",
peertube-1 | "message": "password authentication failed for user \"yNUnREWTp8ZAhYutGHn\"",
peertube-1 | "name": "SequelizeConnectionError",
peertube-1 | "parent": {
peertube-1 | "stack": "error: password authentication failed for user \"yNUnREWTp8ZAhYutGHn\"\n at Parser.parseErrorMessage (/app/node_modules/pg-protocol/dist/parser.js:283:98)\n at Parser.handlePacket (/app/node_modules/pg-protocol/dist/parser.js:122:29)\n at Parser.parse (/app/node_modules/pg-protocol/dist/parser.js:35:38)\n at Socket.<anonymous> (/app/node_modules/pg-protocol/dist/index.js:11:42)\n at Socket.emit (node:events:517:28)\n at addChunk (node:internal/streams/readable:368:12)\n at readableAddChunk (node:internal/streams/readable:341:9)\n at Readable.push (node:internal/streams/readable:278:10)\n at TCP.onStreamRead (node:internal/stream_base_commons:190:23)",
peertube-1 | "message": "password authentication failed for user \"yNUnREWTp8ZAhYutGHn\"",
peertube-1 | "length": 115,
peertube-1 | "name": "error",
peertube-1 | "severity": "FATAL",
peertube-1 | "code": "28P01",
peertube-1 | "file": "auth.c",
peertube-1 | "line": "326",
peertube-1 | "routine": "auth_failed"
peertube-1 | }
peertube-1 | }
peertube-1 | }
peertube-1 exited with code 0
peertube-1 | [domain:9000] 2025-02-06 11:35:17.612 info: Using following configuration file hierarchy: /app/config/default.yaml -> /app/support/docker/production/config/production.yaml -> /app/support/docker/production/config/custom-environment-variables.yaml.
peertube-1 | [domain:9000] 2025-02-06 11:35:17.731 error: Unable to connect to PostgreSQL database. {
peertube-1 | "err": {
peertube-1 | "stack": "SequelizeConnectionError: password authentication failed for user \"yNUnREWTp8ZAhYutGHn\"\n at Client._connectionCallback (/app/node_modules/sequelize/lib/dialects/postgres/connection-manager.js:145:24)\n at Client._handleErrorWhileConnecting (/app/node_modules/pg/lib/client.js:326:19)\n at Client._handleErrorMessage (/app/node_modules/pg/lib/client.js:346:19)\n at Connection.emit (node:events:517:28)\n at /app/node_modules/pg/lib/connection.js:116:12\n at Parser.parse (/app/node_modules/pg-protocol/dist/parser.js:36:17)\n at Socket.<anonymous> (/app/node_modules/pg-protocol/dist/index.js:11:42)\n at Socket.emit (node:events:517:28)\n at addChunk (node:internal/streams/readable:368:12)\n at readableAddChunk (node:internal/streams/readable:341:9)\n at Readable.push (node:internal/streams/readable:278:10)\n at TCP.onStreamRead (node:internal/stream_base_commons:190:23)",
peertube-1 | "message": "password authentication failed for user \"yNUnREWTp8ZAhYutGHn\"",
peertube-1 | "name": "SequelizeConnectionError",
peertube-1 | "parent": {
peertube-1 | "stack": "error: password authentication failed for user \"yNUnREWTp8ZAhYutGHn\"\n at Parser.parseErrorMessage (/app/node_modules/pg-protocol/dist/parser.js:283:98)\n at Parser.handlePacket (/app/node_modules/pg-protocol/dist/parser.js:122:29)\n at Parser.parse (/app/node_modules/pg-protocol/dist/parser.js:35:38)\n at Socket.<anonymous> (/app/node_modules/pg-protocol/dist/index.js:11:42)\n at Socket.emit (node:events:517:28)\n at addChunk (node:internal/streams/readable:368:12)\n at readableAddChunk (node:internal/streams/readable:341:9)\n at Readable.push (node:internal/streams/readable:278:10)\n at TCP.onStreamRead (node:internal/stream_base_commons:190:23)",
peertube-1 | "message": "password authentication failed for user \"yNUnREWTp8ZAhYutGHn\"",
peertube-1 | "length": 115,
peertube-1 | "name": "error",
peertube-1 | "severity": "FATAL",
peertube-1 | "code": "28P01",
peertube-1 | "file": "auth.c",
peertube-1 | "line": "326",
peertube-1 | "routine": "auth_failed"
peertube-1 | }
peertube-1 | }
peertube-1 | }
^CGracefully stopping... (press Ctrl+C again to force)
by the way everytime I do a modification in either the env or compose file (or both) I run: docker compose down, docker system prune -a and docker compose up again.
So. The postgre is made successfully, but then the peertube container cannot connect to it.
What did I try?:
- Tried swapping domain name with WAN IP. The domain name is pointing to my IP. Also in nginx, I have the domain name pointing to my IP and port 9000. My port IS exposed (still)
- I have tried with dummy user / pass in postgre config in case they were too complex
- I have tried also commenting the last section like this:
networks:
default:
external: true
name: ext
enable_ipv6: false
#ipam:
#driver: default
#config:
#- subnet: 172.26.0.0/16
#- subnet: fdab:e4b3:21a2:ef1b::/64
But same behavior
I am getting closer everytime time… I feel like I can almost reach it.
So, once again, I ask for your help and your patience to deal with a newbie of my caliber.
How can I get around
user specified IP address is supported only when connecting to networks with user configured subnets
without, ideally, having to redo my whole docker network management. All my other containers are working nicely.
Thank you very much!