Compare commits
8 Commits
8660793ec2
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 9b890529d1 | |||
| 63f7d3e3bf | |||
| 0d2c7660fb | |||
| 804bf3e964 | |||
| aa74cf42f8 | |||
| 521d08969b | |||
| 74ba296cda | |||
| a5fb874a20 |
1
.gitignore
vendored
1
.gitignore
vendored
@@ -1 +1,2 @@
|
||||
venv
|
||||
docker-compose.dev.yaml
|
||||
@@ -1,6 +1,7 @@
|
||||
FROM python:3.14.7-alpine3.24
|
||||
|
||||
|
||||
RUN apk update && \
|
||||
apk add ipmitool
|
||||
|
||||
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||
PYTHONUNBUFFERED=1
|
||||
|
||||
79
README.md
79
README.md
@@ -1,2 +1,81 @@
|
||||
# PBSupper
|
||||
|
||||
## Description
|
||||
|
||||
This program/container automatically powers on a remote server via 623/UDP to iDRAC, and likely other BMCs, connects to Proxmox Backup Server via SSH, so 22/TCP but this can be customized, creates and runs a sync-job, then shuts down PBS.
|
||||
|
||||
|
||||
## Why?
|
||||
|
||||
I wanted to automate waking up my server, running a sync-job and shuting down said server. iDRAC6/7 doesn't have that option so I've built one.
|
||||
|
||||
|
||||
|
||||
## Configurations
|
||||
|
||||
Here is a description of the most necessary and all the other environment variables
|
||||
|
||||
|
||||
### Necessary envs
|
||||
|
||||
#### BMC envs
|
||||
|
||||
* IDRAC -- IP address of the remote BMC **(Default: None)**
|
||||
* IDRAC_USR -- Username of the remote BMC. This user needs to have permission to power on/off the server as well as check the power status **(Default: None)**
|
||||
* IDRAC_PWD -- Password of the IDRAC_USR user **(Default: None)**
|
||||
|
||||
#### Remote Server envs
|
||||
|
||||
* HOST -- IP address, or FQDN, of Proxmox Backup Server. It needs to listen for SSH connections **(Default: None)**
|
||||
* HOST_USR -- Username of the user on Proxmox Backup Server **(Default: root)**
|
||||
* HOST_USR_PWD -- Password for the username specified in **HOST_USR** **(Default: None)**
|
||||
|
||||
If you want to use SSH-keys, use the following environment variables and leave **HOST_USR_PWD** empty
|
||||
|
||||
* HOST_USR_SSHKEY -- Private ssh-key that will be user to connect to PBS. Leave empty if using username+password **(Default: None)**
|
||||
* HOST_USR_SSHKEY_PASS -- Passphrase for the ssh-key. Leave empty if key is passwordless **(Default: None)**
|
||||
|
||||
|
||||
#### PBS envs
|
||||
|
||||
* LOCAL_DATASTORE -- Name of the datastore on the PBS that is passed in **HOST** variable **(Default: None)**
|
||||
* REMOTE_DATASTORE -- Name of the datastore that the local datastore will sync to/from **(Default: None)**
|
||||
* REMOTE_PBS -- IP address, or FQDN, of the remote PBS that the local PBS will sync to/from **(Default: None)**
|
||||
|
||||
|
||||
#### Container-specific envs
|
||||
|
||||
* RUN_SCHEDULE -- At what time should the server be woken up. Written in Cron notation **(Default: "45 17 * * 5")**
|
||||
* RUN_SCHEDULE_TZ -- Timezone in which container is. Set to your local timezone as otherwise the wake-up time might be inaccurate. **(Default: "Europe/Stockholm")**
|
||||
|
||||
|
||||
### Other environment variables
|
||||
|
||||
Here are the other environment variables that can be set
|
||||
|
||||
|
||||
* HOST_SSH_PORT -- SSH port at which PBS is listening on **(Default: 22)**
|
||||
|
||||
* SYNC_DIR -- Direction of the sync-job. **(Default: push)**
|
||||
* PBS_RATE_OUT -- Speed at which sync is *sent*. Reffer to [PBS documentation](https://pbs.proxmox.com/docs/managing-remotes.html#bandwidth-limit) **(Default: None)**
|
||||
* PBS_RATE_IN -- Speed at which sync is *received*. Reffer to [PBS documentation](https://pbs.proxmox.com/docs/managing-remotes.html#bandwidth-limit) **(Default: None)**
|
||||
* PBS_TASK_LIMIT -- How many tasks should be analyzed. Best to leave it at default **(Default: 50)**
|
||||
* PBS_MAX_RUNTIME_MIN -- Max time which PBS might run after the sync-job. Usually it is caused by Verification jobs or Garbage collection jobs. If a *aptupdate*, *reader* or other tasks are running, then MAX_RUNTIME might be longer as these tasks are critical. **(Default: 180)**
|
||||
* PBS_MAX_SHUT_MIN -- Max time which PBS has for a safe shutdown. This variable is used after a *shutdown* command is sent to PBS server. **(Default: 10)**
|
||||
* AUTO_ADD_NAME -- This is a name that is set as sync-job id. **(Default: "autoBackupScript_one-shot")**
|
||||
|
||||
|
||||
* RUN_ONE_SHOT -- If the script should only be ran once. Do not set/use unless you have specific usecase. **(Default: False)**
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
 
|
||||
|
||||
|
||||
|
||||
|
||||

|
||||
@@ -1,18 +1,18 @@
|
||||
---
|
||||
|
||||
# sudo crontab -e
|
||||
# 0 2 * * * cd /path/to/docker-compose && /usr/bin/docker compose up -d >> /var/log/pbsupper-docker-compose.log 2>&1
|
||||
|
||||
services:
|
||||
pbsupper-1:
|
||||
container_name: pbsupper
|
||||
user: 1600:1600
|
||||
image: tea.shupogaki.org/yuruc3/pbsupper:v0
|
||||
image: tea.shupogaki.org/yuruc3/pbsupper:v0.2.1
|
||||
environment:
|
||||
LOCAL_DATASTORE: somethingSomething
|
||||
REMOTE_DATASTORE: somethingSomething
|
||||
REMOTE_PBS: somethingSomething
|
||||
|
||||
RUN_SCHEDULE: "45 10 * * SUN"
|
||||
|
||||
HOST: somethingSomething
|
||||
HOST_SSH_PORT: somethingSomething
|
||||
HOST_USR: somethingSomething
|
||||
|
||||
32
main.py
32
main.py
@@ -32,6 +32,7 @@ AUTO_ADD_NAME: Final[str] = os.getenv("AUTO_ADD_NAME", "autoBackupScript_one-sho
|
||||
|
||||
RUN_SCHEDULE: Final[str] = os.getenv("RUN_SCHEDULE", "45 17 * * 5")
|
||||
RUN_SCHEDULE_TZ: Final[str] = os.getenv("RUN_SCHEDULE_TZ", "Europe/Stockholm")
|
||||
RUN_ONE_SHOT: Final[bool] = (os.getenv("RUN_ONE_SHOT", "false").lower() == "true")
|
||||
|
||||
|
||||
|
||||
@@ -160,21 +161,21 @@ def theCodeBasically():
|
||||
# print(epoch_time)
|
||||
|
||||
# Check if power is on
|
||||
PWR_STATE = subprocess.run(["ipmitool", "-I", "lanplus","-H", IDRAC,"-U", IDRAC_USR,"-P", IDRAC_PWD,"chassis", "power", "status",],capture_output=True,text=True,)
|
||||
PWR_STATE = subprocess.run(["/usr/sbin/ipmitool", "-I", "lanplus","-H", IDRAC,"-U", IDRAC_USR,"-P", IDRAC_PWD,"chassis", "power", "status",],capture_output=True,text=True,)
|
||||
|
||||
if PWR_STATE.returncode != 0:
|
||||
raise RuntimeError(f"ipmitool failed: {PWR_STATE.stderr.strip()}")
|
||||
raise RuntimeError(f"/usr/sbin/ipmitool failed: {PWR_STATE.stderr.strip()}")
|
||||
|
||||
if "off" in PWR_STATE.stdout.lower():
|
||||
subprocess.run(["ipmitool", "-I", "lanplus","-H", IDRAC,"-U", IDRAC_USR,"-P", IDRAC_PWD,"chassis", "power", "on",],check=True,)
|
||||
subprocess.run(["/usr/sbin/ipmitool", "-I", "lanplus","-H", IDRAC,"-U", IDRAC_USR,"-P", IDRAC_PWD,"chassis", "power", "on",],check=True,)
|
||||
|
||||
|
||||
# PWR_STATE = subprocess.run(["ipmitool", "-I", "lanplus", "-H", IDRAC, "-U", IDRAC_USR, "-P", IDRAC_PWD, "chassis", "power", "status"], capture_output=True)
|
||||
# PWR_STATE = subprocess.run(["/usr/sbin/ipmitool", "-I", "lanplus", "-H", IDRAC, "-U", IDRAC_USR, "-P", IDRAC_PWD, "chassis", "power", "status"], capture_output=True)
|
||||
|
||||
# try:
|
||||
# # power on if needed
|
||||
# if "off" in str(PWR_STATE.stdout.decode()[-4:]):
|
||||
# subprocess.run(["ipmitool", "-I", "lanplus", "-H", IDRAC, "-U", IDRAC_USR, "-P", IDRAC_PWD, "chassis", "power", "on"]) # , capture_output=True
|
||||
# subprocess.run(["/usr/sbin/ipmitool", "-I", "lanplus", "-H", IDRAC, "-U", IDRAC_USR, "-P", IDRAC_PWD, "chassis", "power", "on"]) # , capture_output=True
|
||||
# except Exception as e:
|
||||
# print("Most likely wrong password, username or hostname has been passed for idrac.\nExiting\n")
|
||||
# sys.exit(1)
|
||||
@@ -345,6 +346,10 @@ def theCodeBasically():
|
||||
|
||||
|
||||
(stdin, stdout, stderr) = ssh.exec_command(f"proxmox-backup-manager sync-job run {AUTO_ADD_NAME}")
|
||||
|
||||
# (stdin, stdout, stderr) = ssh.exec_command(
|
||||
# f"nohup proxmox-backup-manager sync-job run {AUTO_ADD_NAME} >/var/log/{AUTO_ADD_NAME}.log 2>&1 </dev/null &"
|
||||
# )
|
||||
# ssh.exec_command(f"proxmox-backup-manager sync-job run {AUTO_ADD_NAME}")
|
||||
# print("here it runs the sync-job")
|
||||
# print(stdout.read().decode())
|
||||
@@ -354,6 +359,9 @@ def theCodeBasically():
|
||||
|
||||
if syncJobRunExitCode != 0:
|
||||
raise RuntimeError(f"Sync job failed: {stderr.read().decode()}")
|
||||
# raise RuntimeError(f"Failed to start sync job: {stderr.read().decode()}")
|
||||
else:
|
||||
print("Sync successful")
|
||||
|
||||
# Give some time for PBS to "start" the sync-job task
|
||||
time.sleep(10)
|
||||
@@ -377,6 +385,11 @@ def theCodeBasically():
|
||||
while currenttasks:
|
||||
|
||||
print("Waiting 1 minute before checking tasks again")
|
||||
|
||||
print("Remaining jobs:")
|
||||
for job in currenttasks:
|
||||
print(f"Task {job} Amount {currenttasks[job]}")
|
||||
|
||||
time.sleep(60)
|
||||
|
||||
currenttasks = getCurrentTasks(ssh, limitOfTasks=PBS_TASK_LIMIT)
|
||||
@@ -384,6 +397,7 @@ def theCodeBasically():
|
||||
|
||||
if time.monotonic() - SYNC_JOB_DATE >= (PBS_MAX_RUNTIME_MIN * 60):
|
||||
# print(SYNC_JOB_DATE - time.monotonic())
|
||||
|
||||
# Forcefully skip if these tasks are running. reader task is, I think, PVE backup
|
||||
# Also otherjob is something else and it is better to wait if there is one otherjob
|
||||
if (
|
||||
@@ -412,15 +426,17 @@ def theCodeBasically():
|
||||
print("PBS is powered off")
|
||||
|
||||
else:
|
||||
subprocess.run(["ipmitool", "-I", "lanplus", "-H", IDRAC, "-U", IDRAC_USR, "-P", IDRAC_PWD, "chassis", "power", "off"])
|
||||
subprocess.run(["/usr/sbin/ipmitool", "-I", "lanplus", "-H", IDRAC, "-U", IDRAC_USR, "-P", IDRAC_PWD, "chassis", "power", "off"])
|
||||
|
||||
# sys.exit(0)
|
||||
|
||||
|
||||
if RUN_ONE_SHOT:
|
||||
theCodeBasically()
|
||||
exit(0)
|
||||
|
||||
|
||||
|
||||
scheduler = BlockingScheduler()
|
||||
scheduler = BlockingScheduler(timezone=RUN_SCHEDULE_TZ)
|
||||
scheduler.add_job(
|
||||
theCodeBasically,
|
||||
CronTrigger.from_crontab(
|
||||
|
||||
@@ -20,7 +20,7 @@ data:
|
||||
# Alternatively you can also use this option instead of PRIVATE_KEY: 123...
|
||||
|
||||
stringData:
|
||||
PRIVATE_KEY: |
|
||||
HOST_USR_SSHKEY: |
|
||||
-----BEGIN OPENSSH PRIVATE KEY-----
|
||||
123...
|
||||
-----END OPENSSH PRIVATE KEY-----
|
||||
@@ -48,11 +48,13 @@ data:
|
||||
PBS_TASK_LIMIT: somethingSomething
|
||||
PBS_MAX_RUNTIME_MIN: somethingSomething
|
||||
PBS_MAX_SHUT_MIN: somethingSomething
|
||||
AUTO_ADD_NAME : somethingSomething
|
||||
AUTO_ADD_NAME: somethingSomething
|
||||
RUN_SCHEDULE: "45 10 * * SUN"
|
||||
|
||||
---
|
||||
apiVersion: batch/v1
|
||||
kind: CronJob
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
|
||||
metadata:
|
||||
name: pbsupper
|
||||
namespace: CHANGE_ME
|
||||
@@ -61,20 +63,20 @@ metadata:
|
||||
idrac: v7
|
||||
|
||||
spec:
|
||||
schedule: "*/15 * * * *"
|
||||
timeZone: "Europe/Stockholm"
|
||||
concurrencyPolicy: Forbid
|
||||
replicas: 1
|
||||
|
||||
successfulJobsHistoryLimit: 3
|
||||
failedJobsHistoryLimit: 3
|
||||
|
||||
jobTemplate:
|
||||
spec:
|
||||
backoffLimit: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: pbsupper
|
||||
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: pbsupper
|
||||
server: Server-1
|
||||
idrac: v7
|
||||
|
||||
spec:
|
||||
restartPolicy: Never
|
||||
automountServiceAccountToken: false
|
||||
|
||||
securityContext:
|
||||
@@ -83,7 +85,7 @@ spec:
|
||||
|
||||
containers:
|
||||
- name: pbsupper
|
||||
image: tea.shupogaki.org/yuruc3/pbsupper:v0
|
||||
image: tea.shupogaki.org/yuruc3/pbsupper:v0.2.1
|
||||
imagePullPolicy: IfNotPresent
|
||||
|
||||
envFrom:
|
||||
|
||||
Reference in New Issue
Block a user