Project Ideas

Core Infrastructure 🐣

Beginner Projects

⭐️Centralized authentication
Problem: Servers and services have their own separate user databases. Adding users, deleting users, and changing passwords has to be done in multiple places.
Solution: A centralized database of user information with a way of authenticating against it.
Example technologies: LDAP via FreeIPA (requires Fedora or RHEL), Samba, or OpenLDAP. SSO via Keycloak,Authentik, or Authelia Questions: LDAP or SSO?
Progress: FreeIPA was installed on leatherhead, client installed on donatello, ssh login via password or key is working.
ISSUES: LDAP authentication isn't working. leatherhead's networking/port forwarding situation is not setup right. Setting up a directory services replica would be good and interesting. A backup scheme is needed.

⭐️Local DNS
Problem: To connect with equipment and services over the network users have to look up IP Addresses on table maintained by hand instead of just using names. It's a problem for the sysadmin group, and it's a problem for HM too.
Solution: a local DNS server that supports dynamic name registration
Possible technologies: BIND9 or Technitium Questions: Use a hackmanhattan.com subdomain like sagroup.hackmanhattan.com? Implement a canary domain? Or otherwise deal with browser DNS over HTTPS?
Progress: an unbound DNS server was installed on april for the subdomain sagroup.hackmanahttan.com. As part of the FreeIPA install leatherhead runs a BIND DNS server with the records necessary for FreeIPA's operation.
Issues: We should probably replace unbound on april with BIND secondary zone both to allow FreeIPA to operate from a directory service replica without leatherhead having to be up.

⭐️VPN
Problem: System administrators would like to be able to access the sysadmin group's equipment from outside HM.HM leadership would like that to happen in a secure and energy efficient manner
Solution: a host-to-network (remote-access) VPN perhaps using a Raspberry PI or other small platform to minimize idle energy use Possible technologies: Wireguard, OpenVPN, RaspberryPi, PiVPN
Questions: Does Wireguard or OpenVPN better integrate with whatever centralized authentication system we implement? Can we restrict access to just sysadmin group equipment?yste Do we want to? Are we going to conflict with HM's VPN implementation?
Progress: Wireguard has been installed on april. One person said the were able to connect remotely to april and start and ssh session. Issues: Other people have not been able to connect. Also does routing through april and to other equipment work? How about DNS settings for the remote tunnel?

⭐️NTP:
Problem: Computer clocks drift, but network things need synchronization and accurate timestamps.
Solution: a local NTP server to fetch accurate time from a reliable source and synchronize our local machines, on port 123.
Doubts: can an always-on service like NTP be run on a raspberry pi
Progress: chrony has been installed on all the machines.

⭐️Configure Router to hand out reserved IP addresses to each server. Problem: currently ip's are dynamically assigned with a long lease; in principle we are one reboot away from the Wiki ip addresses being inaccurate. Progress: All the main server addresses and the some of the iDRAC addresses have been reserved on the router. Issues: The rest of the iDRAC's addresses should be reserved.

🐣Figure out why "wall" command isn't working
sudo wall should write a message out to all currently logged in users - but right now, only if you are logged in locally via the terminal do you see wall messages sent by a sudo user. However, ssh users won't see wall messages. We are running Debian 13 (trixie), let's figure out why this is happening. Currently seen with ssh users using GNOME on rafael. What needs to be done:
1. Document similar or different behavior of sudo wall on other servers
2. Attempt other terminal types across servers to determine if this is terminal-type restricted

🐣Web Server
request: host a password manager

🐣Install a certificate for HTTPS
Problem: When we implement a web app, either it runs over HTTP, which causes modern browsers to complain about it being insecure, or it runs over HTTPS with a self-signed certificate, which browsers complain about.
Solution: Get a certificate for a web server that is signed by a public certificate authority that browsers recognize by default.
Possible technologies: Let’s Encrypt, Web Server, Public and local DNS. What DNS names to we want to use? Can we implement automated renewals?

🪛Router Problem: We have a Cisco router it might be fun to play with or even use for something, but we don’t know if it works or how it’s configured. Solution: Connect up to it, reset it to its default configuration, mount it in the rack, connect it to the switch, set login credentials to the standard password.
Possible technologies: serial cable, USB to serial adapter.

🪛Set up remote logging for servers’ I.D.R.A.Cs
Problem: Each server's Integrated Dell Remote Access Controller keeps a log of events and anomalies, but administrators have to log into each server's I.D.R.A.Cs to view each one.
Solution: Configure the I.D.R.A.Cs to send logs to a central system

🪛Centralized Logging Problem:
Systems (servers, software, etc. generate log files of of events and anomalies, but administrators have to log into each server and view each one.
Solution: Send logs over the network for collection at a single point
Possible technologies: rsyslog, Elasticsearch, OpenSearch

🪛Make a diskless workstation system
Problem: HM sometimes get donations of laptops of desktops with their drives removed as security measure. It would be useful to be able to use them without further investment. It might also be desireable to take laptops or desktops even with drives and boot them into different "personalities" e.g. Linux, WIndows, CAD workstation, etc. to use in a class or workshop.
Solution: Boot over the network
Possible Technologies: PXE Boot, TFTP, Ventoy Server, LTSP or similar
Resources: Archwiki has a good page on the subject Questions: Can we boot a Raspberry Pi via PXE? Can we boot Windows?
Progress: set up on[[raphael]. See the Diskless Workstations progress page for details.
Issues: need an image that will actually fully boot from the network.

🪛Private cloud file storage (aka Google Drive)
NextCloud

🪛Monitoring system
Problem: Finding out whether a service or host is up and responding to requests is a manual process spread out over multiple tools. Diagnosing the cause of a problem often involves manual running tests at various points in the data flow chain.
Solution: Automate tests and collect the data into a central dashboard.
Possible technologies: Nagios, Inciga.
Progress: check_mk was installed on raphael(?)
Issue: Doesn't work

🪛VOIP
Voicemail, Phone Tree
Possible Technolgies: Asterisk

🪛Compute cluster
Distributed computation of processor intensive tasks such as Blender animations.

🪛Multiplayer Game Server
Possible technologies: Nakama, Godot

🪛Enhance Server Security
SELinux, AppArmor, Polkit, seatd

🪛Provisioning/Configuration Management/IaC

Ansible, OpenTofu, Pulumi

🪛High Availability Clustering

🪛Private Cloud Infrastructure
Possible technologies: OpenStack, OpenShift

🪛Project Management
Possible technologies: OpenProject

🪛IP Camera system

🪛Containerization

🪛Change management system

Resolved

🪛Install a Wiki
Problem: We might wish to store sysadmin group's documentation on our own wiki rather that HackManhattan's. Or mess around with different wiki software.
Solution: Locally host a wiki application Possible technologies: MediaWiki (what Wikipedia and HM use), DokuWiki, BookStack