"The Cloud"

[Chris Hogg]

The local history archive where I volunteer is proposing to put all it's files on the cloud. I know very little about the cloud. As far as my knowledge goes, rather than having your computer storage in your own PC or laptop on an SSD or HDD, it's like having your data on someone else's computer that can be anything from next door to thousands of miles away, so I have questions.

How secure is it against 'bad actors' who may want to steal information, or simply delete it or infect it out of malice? Is it more secure than a domestic computer with reasonable anti-virus and firewall protection?

One of the claims being made in our case is that it will reduce our carbon footprint. While I can see that if our particular cloud computer is based in Iceland for example, where all the electricity is generated by renewables, then we may indeed reduce our carbon footprint, but I guess in general that won't be the case. It may reduce our electricity bills a fraction (we have one file server and six elderly PC's in use, mornings only, four days per week, so not many KWh's), but I guess that would be offset against charges to use the cloud, and in most cases (Iceland excepted) that electricity would not be renewable, just not generated in the UK. So is it actually 'green'? I see the Beeb, normally an ardent supporter of all things aimed at reducing CO2, is critical of the cloud.

and Is it backed up anywhere (if so, where), so that if the computer holding it is destroyed either deliberately or by some natural event (Iceland gets volcanoes that pop up in unexpected places!), there are copies elsewhere in another part of the cloud that can be accessed to restore the original data?

[Joe]

Nobody has any way of knowing. One would hope that rented server space should be extremely well protected, but on the other hand, it will be a bigger target and may be vulnerable to attacks from other users. Even Microsoft has been hacked.

I really wouldn't take that aspect into account. It's your data that is important. Nobody's going to pay you for being some unquantifiable value of 'green'.

You might examine a proposed contract carefully, to see if the hosting company will acquire any rights to the data. Once upon a time, customers' email was considered confidential, and email hosts made a point of saying the email was guaranteed private and was never read by the host. After some years, some people started noticing they were getting adverts connected with their emails and it turned out that many hosts had quietly dropped that guarantee.

Possibly your data has no commercial value, but some of it may be covered by someone's copyright.

It certainly ought to be, and the details will be part of the contract, or at least mentioned in the marketing. If I was doing this I would probably try to maintain my own backup copy on hard drives.

[Tim Streater]

I would have said that the other thing to beware of is where the data is actually hosted and what path it takes to get from you to there. The Yanks have form for being officious that if the data is hosted in the US, or perhaps even only passes through the US to get to the server, they give themselves the right to have access to it if ordered by some tuppeny-ha'penny court in Armpit, Nebraska or similar.

[Cursitor Doom]

Indeed. If you upload your data to the Cloud you need to be sure there's nothing in it which is personal/confidential or which could give rise to adverse legal consequences if made known to any third party, because you're effectively sharing it with God knows who. There's no such thing as a free lunch so why would any rational enterprise want to spend good money on electricity to help you out and get nothing back in return as so many of these Cloud storage services do. They often offer a Gigabyte or more storage for free. Why would they do that? What are they getting out of it that you don't know about?

[Adrian]

In message [e-mail]>, Tim Streater [e-mail]> writes

Another thing to bear in mind is that it is not unknown for cloud systems to disappear (hosting companies cease trading). If the only place that you've got your data stored is on that cloud, then you are stuffed.

Adrian

[John Rumm]

Cloud storage can be fairly secure if access to it is controlled carefully. On the bright side it is probably better protected against equipment failure. While it is unlikely that one of the big cloud providers will easily be hacked directly and the data compromised, you need to be vigilant that the users of it do not compromise the security by getting phished or socially engineered etc.

The first question I suppose is what are you planning to move to the cloud? Just data, or in effect the whole server?

Most cloud providers have the capabilities to run virtual machines in the cloud, and they often sit upon sophisticated platforms that can shift VMs about onto physical severs dynamically based on the load, amount of traffic, time of day etc. The energy requirements for that flexibility may be higher, but they can also be shared between more users.

If power saving is of particular concern, you may be able to do something about that fairly easily locally... A low power small form factor PC would be able to run all of your current PCs as virtual machines on the one physical computer, then something very cheap and low power like a batch of Raspberry pi computers could serve as a Remote Desktop Protocol client to access them. (or better still, get the users to use their own hardware and lekky for access!)

If you opt for backup services then yes, but don't assume it will be by default. (it will be fault tolerant storage - so you are unlikely to lose data due to a disk failure, but that does not protect you from user error, deleting or overwriting good data with bad.

The term "cloud" is a bit nebulous (yeah, I know :-)

It could just mean a lone physical server sat in a data centre, but more likely these days it will be a virtual server, storage for which will likely be on some kind of shared fault tolerant storage device (SAN etc), and the actual virtual machine image will be deployed with many others onto physical hardware. So a server going t*ts up or suffering a disk failure is unlikely to even be noticeable at your end, and things can be automatically migrated to working hardware.

Cloud storage normally allows multiple ways to download data - many also offer the option of sending out physical media for quick disaster recovery[1].

[1] "Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway." –Andrew Tanenbaum, 1981

[David Wade]

I have copies on multiple computers, as well as the cloud.

Not much different. Modern social-engineered attacks are designed to persuade you to run something with elevated privileges which encrypts your data.

I would say this is "green washing"

That depends on your cloud contract. I would say in the case of the free ones no, but then how good are your existing backups? Have you ever done a cold restore? Can you re-build the server Operating System? In the event of a fire or lightening or power surge are you good to go.

I use one-drive so all my PCs have copied of the files. I also do weekly backups to a small NAS box so have multiple copies.

Dave

[Sam Plusnet]

You could & should go through the contract in fine detail when it's first in place. However (as an example) gmail send me an email from time to time which mentions that there have been a few changes to their T&Cs. There will be a link to the new T&Cs, but rarely is there a detailed explanation of what the changes are in a form that I can easily digest.

In other words, your "Cloud" contract might be everything you want on day one, but changes at some later point might come back to bite you.

[Thomas Prufer]

Sounds a bit like someone has heard buzzwords and is making suggestions without knowing what work or problems they entail?

If carbon footprint is a main reason for doing it: get an electric contract with a smaller carbon foot print, i.e. one that uses lots of renewables. Job done...

Also, it sounds as if only the file server would move? Then getting newer "low power" PCs locally may be a better option to reduce power use. A small brick containing all the works, plus keyboard, mouse and screen, done.

Thomas Prufer

[N_Cook]

So you have all your eggs in one basket. I would back up on multiple DVDs written at very slow speeds for the deepest of laser pits, stored in hermetically sealed correct ,maybevacuum, environments to minimise corrossion of the Aluminium , in different sites in case of flood or fire and also multiple external HD stored on different sites and also multiple datasticks. Even the pro York Data Services the premier UK store of UK archaeology data, you cannot find how they store it. Assumed to be RAID array which can be infected with malware and brought down.

[Adam Funk]

Good one!

[David Wade]

That would be nice, but challenging. I have nearly 400gb of data on my Microsoft OneDrive so that is a mere 40 DVDs so unless I had multiple machines I would pretty much have to dedicate my life to backing up my data.

Any writeable storage can be infected by Malware, even your DVDs. It can lurk for ages before it activates and corrupts your live system. The data is unlikely to be on a traditional RAID array. I would suspect its some kind of SAN (Storage Area Network) hopefully with multiple copies.

If you build things properly you can have data replicated in real time across multiple sites. If you also have multiple snapshots then you can roll back the data after malware infections, that is assuming there is physical separation of the data stored in the SAN, and the "data" used to load the software into the SAN which always used to be the case, but its been a while since I worked with such stuff.

as with anything it can be expensive, but the ransomware may be even more expensive.

oh that brings back memories from working at NERC Bidston where one of the things we did was to track mean sea level.

[Theo]

It is a question worth asking as to what this service actually provides.

In this case it's quite likely that the service being rented is 'data storage' and not 'servers'. In other words somebody looks after storing the data, and how they do that isn't something you interact with - they must run some servers but where and how they run that isn't something you are exposed to. The service offer you ways to move data in and out.

This can be called 'storage as a service' (STaaS) but can also be combined with other services like online database tools which are more generally 'software as a service' or SaaS.

While it is possible to rent 'servers' and then run your own install of software, that's probably not something you want unless there's a good reason you need to manage the software running on top of the servers that you rent. (cloudy people call this 'infrastructure as a service' or IaaS because you're renting just the hardware, and the software is your own)

The OP is dead right that 'the cloud is someone else's computer' so the questions to ask are about how much care the 'someone' takes with your data and how they manage failures, access and prevent data breaches. And billing, and also what happens if they raise the prices too much and you want to leave.

You will want to work out what the steady-state running costs are likely to be - while they may provide a better service than you could DIY, it may well be more expensive than whatever you are doing at the moment.

(there may be some introductory offer or whatever, but be aware it's another project to migrate data out of a cloud provider so it's not always straightforward to hop over to somebody else - which is why the regular pricing matters)

Theo

[N_Cook]

The 'galling' thing about archiving is, the ancients got it right. Use oak gall ink on goat skin and its perfectly readable 1000 years later, ok now reddish-brown rather than black but excellent contrast.

[N_Cook]

An added reason for the record high sea level reached for Storm Pierrick affecting the south coast. The curves here should on average be coincident

and for other UK ports. The EA has interpreted the 'errors' as an error with the big data NTSLF surge predictor. But luckily in fact its an error with the calibrations of the tide gauges. There is an anomolous increase in UK mean sea level over recent years, beyond global SLR and local postglacial isostatic ground movement. With a couple of NOC researchers currently but looks like due to excess freshwater from Greenland, Baltic influx etc. This 'error' is at least 0.15m and for the 3 days including 08/09 Apr

2024 was an rms error of 0.269m . ie something like 0.27m needed adding to the NTSLF and UKHO tidecurves and so tidetables, otherwise a normal spring tide this week , before any surge was added on top.

[Theo]

Suppose you needed a database.

1. You could install MS Access[*] on a PC under your desk. We'd call that 'on premises' or 'on-prem'. Maybe you'd rather put that PC in a dedicated room in the basement, but that's the same thing. Maintaining everything - the hardware, the software, the power supply, the user accounts, ... is all up to you.

[*] not a recommendation for database software!

1. You could rent a server in a datacentre somewhere, install Windows and MS Access on it yourself. That's Infrastructure as a Service or IaaS. Here the hardware is now somebody else's problem but the data stored on it (and security and backups thereof) is yours. You would access it over the internet not over the LAN but it would work roughly the same way.

2. Or you could pay Microsoft for the MS Access 365 web service. There MS take care of both the hardware and the software and just provide you with a login to the online version of Access. That's Software as a Service (SaaS). MS take care of handling hardware failure, and they also look after the data so if the hardware crashes the data is maintained. They may also maintain availability, so that if the datacentre in Ireland becomes unavailable for some reason, your query gets routed to an alternative server in Germany which will handle it. They also take care of some of the management overhead (eg checking who has access, managing their passwords, etc etc)

As you go through the levels, the provider is doing more work for you. Like-for-like they will cost more, but can often work well if your needs are small (if you only need to store 10GB then paying for 0.01% of someone else's shared server is going to be cheaper than buying a PC and only using

0.01% of it)

What you cede going up the levels is control to do things how you please. OTOH they probably do a better job than you will, and maybe you would rather get on with doing your job and not managing servers.

As I said in my other post there are risks for you in terms of the competence of the cloud provider, their billing and/or price rises, and the complexity of switching provider should you decide to move elsewhere. Which might be set against your competence to manage your own servers and the costs of doing so, plus the risk to the organisation if you go under a bus.

Where you sit on the spectrum will depend on how much technical competency and resources you have, as well as the particular business needs.

Theo

[Joe]

It's not bad as a Rapid Application Development environment for a user front end to a database. The user interface is reasonably good (and was in the days of Access2 on Win3), just don't use it to store the data. Quite a lot can be done without needing to touch program code.

Just to add a note, Access can use a variety of database storage systems, so you could use an in-house Access front end and an outsourced SQL server (or other type) for the data. That gives you the multiple user facility that Access by itself doesn't have (it does in theory but it relies on operating system file locking, which is ...sub-optimal).

It also gives you a non-Access method of getting to the data for troubleshooting, auditing or programmable access such as web applications. It is also easy to dump an entire SQL database (in text format) for your own backup and in an emergency, such as an Internet outage, it can be restored to any SQL server and read.

[Chris Hogg]

Thanks for all the advice and suggestions. I'm told that we will be using MS365, so from what Theo has recently said about it, a lot of my concerns should be taken care of. But still worth my enquiring from the PTB here, if all those points have been taken into considerations. Thanks again.

[Paul]

I can think of a couple choices.

Archive.org (which was recently used to archive a government web site for some country). There might be a charge for doing that, if you're asking for "bulk help". Because they need the money to buy more "storage trailers".

Backblaze (which backs up hard drive sized chunks).

Whereas other kinds of Cloud providers, have a little of everything.

*******

The OP should start by creating a list of requirements, and when asking a question like this, present the summary of the requirements. Maybe an RPi in the basement is enough :-)

I'm kinda curious how big the current (scanned) collection is.

For backups, the restoration may not be real time. With tiered storage, your "web site" can be stored on tape, and the tape robot has to extract the image and make it available on a disk drive (for a limited time). To start your restoration, may involve a 24 hour delay.

Paul

[Paul]

That would be 4 discs out of this 25-pak.

Not nearly so challenging.

"VERBATIM AMERICAS LLC 98915 M-DISC BD-R 100GB 4X INKJET PRINT 25PK"

Verbatim makes it easy, with the five digit product code, to look up the product code on a nearby website.

You will need a BluRay drive with the MDISC logo on it.

That's currently the largest disc I know of (that uses a reasonable storage layer inside).

BluRay was "supposed to be marginal on USB2", which tells you, on a good day, the read-back rate could be as high as 30MB/sec. Maybe Wiki has an exact number. The write is unlikely to be that fast, but it should neatly beat a DVD on write.

Paul