VPS and privacy

toho · May 2020

Is it possible (and legal?) for a provider to access the filesytem of a vps.
On OpenVZ it is probably something like:
cd /share/user4345387/fs

But what about KVM virtualization with encrypted filesystem and preboot authentication ?
Is it possible for the hoster in a default setup to access the unlocked encrypted guest filesystem ?

What about the legal ascpect of this ?

benj0x · May 2020

@toho said:
Is it possible (and legal?) for a provider to access the filesytem of a vps.
On OpenVZ it is probably something like:
cd /share/user4345387/fs

But what about KVM virtualization with encrypted filesystem and preboot authentication ?
Is it possible for the hoster in a default setup to access the unlocked encrypted guest filesystem ?

What about the legal ascpect of this ?

Yep. Just ramdump the KVM server and you'll get the key for the encrypted fs.

Probably illegal. But I guess the many providers around here can clarify.

deank · May 2020

No harm done if no one finds out...

That's how things work in low end segment.

seriesn · May 2020

Is it legal? No. Can it be done, yeap. Should it be done? Unless there is a request made by legal authority, no.

Anon · May 2020

Providers spy on you. The smaller the provider, the more free time he has for espionage.

Anon · May 2020

Funny, but some providers ask you for a password if you open a ticket and ask for a help. Once I was looking for a help, opened a ticket and forgot to disable ssh key to help the provider to get access to my VPS, but the provider just get into my VPS with no problem at all, not having a password, not having a ssh key from me. That's the time when I understood that there's no privacy in modern world. If some providers would like to see the nudes of your girl, he can easily look in your nextcloud

mikho · June 2020

@Anon said:
Providers spy on you. The smaller the provider, the more free time he has for espionage.

I’m a small provider, I don’t have time, nor the interest in poking around your files.
Unless the police comes knocking at my door.

The only time a provider should look at a users files are with the consent of the user, most often to help out on a problem.

sureiam · June 2020

This is why dedicated servers and colocation are the only sure ways to retain privacy. Affordable options (lowend) are atom dedicated servers (preferably with raid 1) and pi colocation.

seriesn · June 2020

@Anon said:
Providers spy on you. The smaller the provider, the more free time he has for espionage.

I mean I get bored at times, but still not interested in seeing what is inside your VM. Chances of reddit having better content is pretty high.

bdl · June 2020

@Anon said:
Providers spy on you. The smaller the provider, the more free time he has for espionage.

"I AM BORED. Time to h4x0r my 31337 kl13n75"

@seriesn said:
I mean I get bored at times, but still not interested in seeing what is inside your VM. Chances of reddit having better content is pretty high.

/r/ryzengonewild

bugrakoc · June 2020

@Anon said:
If some providers would like to see the nudes of your girl, he can easily look in your nextcloud

That's why you should never have nudes in the first place. Never understood the purpose of them tbh.

deank · June 2020

The only absolute safe place is a coffin, for a little while before bugs get in.

vyas · June 2020

@deank said:
The only absolute safe place is a coffin, for a little while before bugs get in.

Or....

InceptionHosting · June 2020

@toho said: Is it possible (and legal?) for a provider to access the filesytem of a vps.

Yes, it is possible and depending on the terms, circumstances and intent, legal.

@toho said: But what about KVM virtualization with encrypted filesystem and preboot authentication ?

Yes but not as trivial to do so.

@toho said: Is it possible for the hoster in a default setup to access the unlocked encrypted guest filesystem ?

Yes possible, not at all trivial.

@toho said: What about the legal ascpect of this ?

See the terms and computer misuse act of the country of the registered business for answers.

I think what everyone needs to understand is that your VPS disk, KVM, Xen, VMware, OpenVZ, virtuozzo, LXC, LXD etc etc is just a file on a bigger server, if that host server could not mount your disk then your VPS would not exist, to begin with.

Hosts have physical access to your server emulated or otherwise.

We can access your file system transparently and silently in most cases, we can also access your RAM making encryption less effective.

If you don't trust your host don't use them, pay more for your own hardware and security solutions, if you can't pay more to own and operate your own equipment and won't trust hosts then don't put it online, to begin with, you can't have it all ways.

In the case someone locks themselves out and decides to create work for a host rather than just putting the server in rescue mode and mounting their own disk and editing their own sshd_conf then what other choice are you giving us, we mount your disk and edit your files as per your request, we obviously would not mount your disk while it is in use as that could cause issues hence asking for an appropriate user/password if you cant do it yourself and the server is accessible via conventional means.

I think I have my next article idea though, a demonstration on KVM/OpenVZ of how secure your files are as it still seems to be a concern and point of wonder despite the thousands of times this conversation has played out online.

seanho · June 2020

Nowadays with Proxmox, virt-manager, vagrant, etc., it's super easy to spin up a few KVMs and/or LXC and see for yourself just how dependent the guest OS is on the host OS, and how easy it is for the host to access secrets in the guest.

yoursunny · June 2020

I have limited trust on VPS.
For OpenVZ, the “serial console” can enter the container without any password. Nothing prevents the provider from doing the same.

While I have my website and Nextcloud there, I keep off the really sensitive stuff.
The SSH private key on VPS is a separate one that is granted minimal GitHub privilege.
Scans of tax forms and passports stay in Dropbox.

vimalware · June 2020

The SSH private key on VPS is a separate one that is granted minimal GitHub privilege.

I felt like taking a minute to chime in regarding ssh keypairs.
Every 'host' should have its own separate private key generated on itself (or off-vps : if you do not trust the user-land ssh-keygen and dependency libs(say openvz from a sketchy new host) )

Your own local workstation/laptops' private keys should not exist anywhere else except inside an encrypted backup.

deank · June 2020

The only truly secure data is one that is offline.

Or have your server in your basement or closet.

If above two options are not possible, dedicated servers are probably the only way with your own cabinet and key.

yoursunny · June 2020

@vimalware said:
Your own local workstation/laptops' private keys should not exist anywhere else except inside an encrypted backup.

3 workstations + 2 phones = 5 entries in authorized_keys
6 servers + 5 online services = 11 places to edit every time I get a new device

OR

primary key for all my workstation and phones, saved in Dropbox, not uploaded to VPS
1 entry in authorized_keys, and 1 place to edit when I get a new device
limited single purpose keys, usually only on one client and can only access one server or online service

Ouji · June 2020

On that topic, how do you guys handle your ssh keys? I have several servers, but one key per client is starting to become a hassle.

seanho · June 2020

Agent. Key represents an identity, not a host https://developer.github.com/v3/guides/using-ssh-agent-forwarding/

FlamingSpaceJunk · June 2020

@sureiam said:
This is why dedicated servers and colocation are the only sure ways to retain privacy. Affordable options (lowend) are atom dedicated servers (preferably with raid 1) and pi colocation.

Maybe. At last year's BSides there was a talk about backdooring BMC chips, like iDRAC, iLOM, and so on. The premise was renting dedis and buying used servers is dangerous because BMCs can be compromised giving attackers a persistent way into the server, and people are better off with a VPS or cloud VM.

It's plausible.

@vimalware said:
I felt like taking a minute to chime in regarding ssh keypairs.
Every 'host' should have its own separate private key generated on itself (or off-vps : if you do not trust the user-land ssh-keygen and dependency libs(say openvz from a sketchy new host) )

Yes. Don't reuse ssh keys. It's like reusing a condom.

Your own local workstation/laptops' private keys should not exist anywhere else except inside an encrypted backup.

They should also have a password to add more security in case they are lost/stolen/exfiltrated.

I go the extra mile and generate key pairs for everything instead of reusing a single pair.

@seriesn said:

@Anon said:
Providers spy on you. The smaller the provider, the more free time he has for espionage.

I mean I get bored at times, but still not interested in seeing what is inside your VM. Chances of reddit having better content is pretty high.

This is the thing. I've been in the IT game a long time with lots of deep access, and I always have better things to do. Most people do. Very few people care about other people's data, and most people's IP is useless to others. Unless you're storing the Colonel's recipe for friend chicken or the original cocaine laced Coca-Cola recipe, your files aren't worth anything to other people.

FlamingSpaceJunk · June 2020

@Ouji said:
On that topic, how do you guys handle your ssh keys? I have several servers, but one key per client is starting to become a hassle.

ssh-agent and a .ssh/config file.

HashKnownHosts yes

Host *.domain.tld
        User user1
        IdentityFile ~/.ssh/domain

Host x1nick
        Host x1.domain.tld
        User accountname
        IdentityFile ~/.ssh/domain_rsa
        IPQoS 0x00

Host git*
        User git

Host gitlab.com
        IdentityFile ~/.ssh/gitlab

Host *
        IdentitiesOnly yes

Alternately, setup FreeIPA and setup Kerberos or add SSH keys there.

seriesn · June 2020

@FlamingSpaceJunk said: Colonel's recipe for friend chicken

I will risk all my reputation for that.

vimalware · June 2020

Take out a couple of hours and create a nice .ssh/config file.
I symlink mine out to my Seafile server's synced copy.

FlamingSpaceJunk · June 2020

@seriesn said:

@FlamingSpaceJunk said: Colonel's recipe for friend chicken

I will risk all my reputation for that.

No joke. It's tasty.

Mason · June 2020

@FlamingSpaceJunk said:
Maybe. At last year's BSides there was a talk about backdooring BMC chips, like iDRAC, iLOM, and so on. The premise was renting dedis and buying used servers is dangerous because BMCs can be compromised giving attackers a persistent way into the server, and people are better off with a VPS or cloud VM.

It's plausible.

I'm confused, isn't the host machine of said VPSes/Cloud VMs susceptible to those same attack vectors? Or is it more of a "let's just trust that the provider knows what they're doing rather than the average unmanaged user"?

FlamingSpaceJunk · June 2020

@Mason said:
I'm confused, isn't the host machine of said VPSes/Cloud VMs susceptible to those same attack vectors? Or is it more of a "let's just trust that the provider knows what they're doing rather than the average unmanaged user"?

Yes, if the host has a BMC chip which is plugged into the network, it's a candidate for this kind of attack.

Unknown persons having complete access to the hardware before you is the problem. It's like taking a turn with the town bike.

It's more about trusting the provider has better security and hygiene. A reputation for renting compromised equipment is bad for business.

Mason · June 2020

@FlamingSpaceJunk said:

@Mason said:
I'm confused, isn't the host machine of said VPSes/Cloud VMs susceptible to those same attack vectors? Or is it more of a "let's just trust that the provider knows what they're doing rather than the average unmanaged user"?

Yes, if the host has a BMC chip which is plugged into the network, it's a candidate for this kind of attack.

Unknown persons having complete access to the hardware before you is the problem. It's like taking a turn with the town bike.

It's more about trusting the provider has better security and hygiene. A reputation for renting compromised equipment is bad for business.

Cheers, thanks for the explanation! Makes sense to me.

sureiam · June 2020

@AnthonySmith said:

@toho said: Is it possible (and legal?) for a provider to access the filesytem of a vps.

Yes, it is possible and depending on the terms, circumstances and intent, legal.

@toho said: But what about KVM virtualization with encrypted filesystem and preboot authentication ?

Yes but not as trivial to do so.

@toho said: Is it possible for the hoster in a default setup to access the unlocked encrypted guest filesystem ?

Yes possible, not at all trivial.

@toho said: What about the legal ascpect of this ?

See the terms and computer misuse act of the country of the registered business for answers.

I think what everyone needs to understand is that your VPS disk, KVM, Xen, VMware, OpenVZ, virtuozzo, LXC, LXD etc etc is just a file on a bigger server, if that host server could not mount your disk then your VPS would not exist, to begin with.

Hosts have physical access to your server emulated or otherwise.

We can access your file system transparently and silently in most cases, we can also access your RAM making encryption less effective.

If you don't trust your host don't use them, pay more for your own hardware and security solutions, if you can't pay more to own and operate your own equipment and won't trust hosts then don't put it online, to begin with, you can't have it all ways.

In the case someone locks themselves out and decides to create work for a host rather than just putting the server in rescue mode and mounting their own disk and editing their own sshd_conf then what other choice are you giving us, we mount your disk and edit your files as per your request, we obviously would not mount your disk while it is in use as that could cause issues hence asking for an appropriate user/password if you cant do it yourself and the server is accessible via conventional means.

I think I have my next article idea though, a demonstration on KVM/OpenVZ of how secure your files are as it still seems to be a concern and point of wonder despite the thousands of times this conversation has played out online.

The most important part of this is trusting your provider. There are many low cost solutions which I pass on.

@FlamingSpaceJunk said:

@sureiam said:
This is why dedicated servers and colocation are the only sure ways to retain privacy. Affordable options (lowend) are atom dedicated servers (preferably with raid 1) and pi colocation.

Maybe. At last year's BSides there was a talk about backdooring BMC chips, like iDRAC, iLOM, and so on. The premise was renting dedis and buying used servers is dangerous because BMCs can be compromised giving attackers a persistent way into the server, and people are better off with a VPS or cloud VM.

It's plausible.

@vimalware said:
I felt like taking a minute to chime in regarding ssh keypairs.
Every 'host' should have its own separate private key generated on itself (or off-vps : if you do not trust the user-land ssh-keygen and dependency libs(say openvz from a sketchy new host) )

Yes. Don't reuse ssh keys. It's like reusing a condom.

Your own local workstation/laptops' private keys should not exist anywhere else except inside an encrypted backup.

They should also have a password to add more security in case they are lost/stolen/exfiltrated.

I go the extra mile and generate key pairs for everything instead of reusing a single pair.

@seriesn said:

@Anon said:
Providers spy on you. The smaller the provider, the more free time he has for espionage.

I mean I get bored at times, but still not interested in seeing what is inside your VM. Chances of reddit having better content is pretty high.

This is the thing. I've been in the IT game a long time with lots of deep access, and I always have better things to do. Most people do. Very few people care about other people's data, and most people's IP is useless to others. Unless you're storing the Colonel's recipe for friend chicken or the original cocaine laced Coca-Cola recipe, your files aren't worth anything to other people.

Back when I co-located (wasn't cost effective ultimately for my needs) I had the idrac port physically disconnected from the network. I needed Idrac access like once every year or so at most. A low priority ticket was enough for someone to plug it in and then unplug it a few days later. Sometimes low tech solutions are the best if they are feasible.

Also on this note people should remember to turn off VNC access after setting up their server. Little extra protection goes a long way.

Your right in that most people in IT don't care about that data. My greatest concern though has always been low level very poorly paid staff outsourced to countries with poor reputation of going after crooks. That foreign phone agent with full access to my credit report and bank accounts for example, who's now also working from home.... Not trying to generalize as it can happen anywhere but personally that's the kinda access that makes me weary.

InceptionHosting · June 2020

@sureiam said: Also on this note people should remember to turn off VNC access after setting up their server. Little extra protection goes a long way.

As a host I can confirm this is excellent advice, I can confirm common VNC port ranges are scanned multiple times daily.

Those that sign up with "change" or "passwords" or "zaq12wsx" or "qazwsx" etc etc the usual ones in the quick 100 list of any basic dictionary attack are usually the ones that get compromised, they change their root password straight away and don't think to change their VNC password or disable VNC.

VPS and privacy

Comments