Best Architecture/platform for new web/apps programming scheme

Greg Paul
Greg Paul used Ask the Experts™
on
We have a proprietary and patented video processing software that we license to others and we use on our own sites.  A couple or our sites/apps get approximately 3 million page views per month and are on the verge of profit.  The plan is to expand/scale the sites we own in the next six to twelve months.  This scaling presents us with a opportunity to update everything in our system.  Even though they were launched only a few years ago, so much has changed that we are forced to look at the whole architecture from hosting to platforms and development environments, etc.  We are truly a garage startup and have to make the most of limited resources while trying to present a high quality solution.  It is difficult and stressful.  We cannot afford to get this wrong thus my appeal to the many experts here. We are starting with a clean slate and very much need to hear from experienced professionals to help us put together the best possible system from the top down.

Server/website -  up to now we have used dedicated hosting.  It appears that VM is now a good and much less expensive option.  I am OK with that.  In speaking with other insiders it seems that more and more people are going toward utilizing Azure or Amazon for hosting.  We do have licensees in Europe who must be served by hardware in Europe due to latency issues.   I think Azure or other allows a much quicker provisioning of another instance in whatever geographical area needed, that we can then customize for the project. For the web back end we are looking at .NET CORE.  Not much interested in Linux based situations due to having a fair amount of available expertise in all things .NET.

ISSUE 1
The primary issue at this point is to decide what platforms we will use for the website, Android and iOS apps.  No real movement can happen until platforms/architecture are decided.  I am an engineer who understands programming in general and has written code but I am not at all up to date on any of the various options are and how they integrate with one another.
  1. What platforms allow the highest amount of transference between the three modes of website, Android app, and iOS app?  We would all hope for a universal programming language that is powerful, fast, and stable but to my knowledge this does not exist.  So then what is the current best combination of languages that cover our needs and allow some amount of shared or easily adapted code.  For example, if we write a module that adds a feature which maintains a log of commute times to work/office in the morning,  where do we first create that feature and how much of the code can be used without major effort in the other two platforms?
  2. How does the stability and reliability of those platforms compare to industry agreed current kings of stability and reliability?  
  3. Take into consideration needs of programmers such as available resources, how hard is it to program in these languages, difficulty of adding updates/features, ability to track and document changes, etc.  

ISSUE 2
Apps - this is where I am completely lost.  A platform(s) that allow the most cross pollenation of code.  Meaning if we code a feature, it will be as simple as possible to make it work correctly on the website and 2 apps.

How to do updates or new releases that only affect the specified area or feature and ensure these changes don’t break other parts of the website or app?  This has been an ongoing issue that can have a tremendous negative affect on customers and is burdensome for admin staff as they both deal with frustrated customers and try to pinpoint all of the negatively affected features, document it, and then get a fix implemented by the programmers and retest everything.  We use revision software and any solution must inclue a strong revisioning program.
  1. Is this a common issue or can we implement architecture and procedures to reduce or eliminate this headache?
  2. What are the most useful methods to reduce these issues?
  3. How much effort/hours are required up front to reduce or eliminate these issues?
  4. Some suggested systems (I know nothing about them) – Appflow, Xamarin.

NOTE:  There are many questions here but I didn't think that asking them as separate questions would be productive because we must take into account the overall design, current frustrations, and desired features of the overall architectures/systems.  If needed I can break this down into more questions to be sure experts from different disciplines get the points that they merit.

Thanks to anyone who has any experience to help us make a better decision and avoid wasted time and limited funds.

Greg
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
David FavorFractional CTO
Distinguished Expert 2018

Commented:
1) You said...

 We cannot afford to get this wrong...

Server/website -  up to now we have used dedicated hosting.  It appears that VM is now a good and much less expensive option.

These 2x statements are incongruent.

Trying to run heavy traffic + also any type of video transcoding inside a VM... will likely be the death of your company.

Stick with bare metal servers for best price/performance.

2) Not much interested in Linux based situations...

If you're looking for optimal long term price/performance, you'll use Linux... as free (cost of Linux) scales forever.

Just avoid using AWS or any other VM system, as AWS is notorious for generating what EE folks like to call surprise bills.

The first month you get a 5 Figure surprise bill from AWS or any other VM type system, you'll wish you'd have stayed with dedicated iron... physical machines...

Also debugging performance issues on Linux systems, to me, is far easier than other OS options.

Also, all ffmpeg dependency libraries natively build on Linux, so for video transcoding conversions between different containers + audio/video codecs is a snap. With other OS options... wow... your transcoding options will either be limited or will take a far bit of time to get working.

3) ISSUE 1 - I think you're talking about a system like https://www.appcelerator.com where you write one set of source then device specific code for Linux (Android) + iOS are generated for you.

There are many systems like this. You'll just have to research + test which works best for you.

4) ISSUE 2 - My guess, from what you've stated, is this isn't really an issue.

You'll write your UI code in something like Appcelerator, then all your transcoding will be done across servers, never mobile devices.

5) From all your questions, might be useful for you to go through old EE questions related to building large video streaming platforms (there have been many recently), then pick a few commenters who provide information you find useful for your project, then hire all of them for hour long consulting calls for each to provide you with a high level design.

My guess... if you take the intersection of all these consulting calls together, you may come up with some fantastic design input.
John TsioumprisSoftware & Systems Engineer

Commented:
With some many views/clients and the rest is not that easy to advise but.....
A VM can handle quite some load...how much is just a matter of power vs money...there are VMs that cost around $3/mo there are VMs that cost hundreds dollars/mo...
All i can say is that .NET core seems a solid choice...while there are a lot of choices circulating around but .NET seems to be able to utilize everything and it does it in just about every platform (even mobile -->Xamarin)
David FavorFractional CTO
Distinguished Expert 2018

Commented:
VM Tip: VMs work by translating entire CPU architectures to pseudo code machines.

This means you'll loose a massive amount of processing bandwidth to this translation process.

For some applications, this massive drop in performance is acceptable.

For video transcoding + serving (which I've done daily for many projects) will fail miserably... well... this will work if you're only transcoding a few videos a week.

If you're planning on running the next YouTube of Vimeo or Wistia, bare metal servers are required.

That said, I run all my projects in LXD containers for all sorts of reasons.

One of the perks of LXD is having a container system, which is... VM-esque... with zero performance penalty...

If this massive performance + streaming 1000s of simultaneous videos streams of cheap hardware is required...

Then stick with LAMP Stacks, rather than .NET tech.
11/26 Forrester Webinar: Savings for Enterprise

How can your organization benefit from savings just by replacing your legacy backup solutions with Acronis' #CyberProtection? Join Forrester's Joe Branca and Ryan Davis from Acronis live as they explain how you can too.

David FavorFractional CTO
Distinguished Expert 2018

Commented:
Suggestion: Start with your numbers

1) Simultaneous streams you expect will be required.

A $100/month machine running LAMP can run 1K-10K streams, so long as you have a good mix of high compression HEVC streams.

2) Number of simultaneous transcodes you'll be running.

Generally the way I arrange this is to run several queues.

a) High priority queue runs at lowest numbered priority, so this queue transcodes first.

b) 3x other low priority queues.

c) Each transcode in #a + #b is set to use all threads, ensuring all CPUs + Disk I/O maxes out saturation.

3) Tip: Always be sure to save the original footage, so each new format which comes on the scene, like h.265/HEVC or some update to your own transcode system, you can quickly re-transcode all footage.
Chinmay PatelChief Technology Ninja
Distinguished Expert 2018

Commented:
Hi Greg,
I am going to give this one a try. I believe (generally I am more than happy to answer the questions), to get some concrete output you will require brain-storming sessions for a couple of weeks.With that said, let us get started.

I am glad you are thinking about Cloud for Compute, Storage and Delivery. I want to highlight that VMs are not the only options - you have many options. You can think about
1. Azure Functions - https://azure.microsoft.com/en-us/services/functions/
2. High Performance Computing - https://azure.microsoft.com/en-us/solutions/high-performance-computing/
3. Batch Rendering (Offered under HPC) - https://docs.microsoft.com/en-us/azure/batch/batch-rendering-service
4. Azure Serverless - https://azure.microsoft.com/en-us/solutions/serverless/

And for rendering, very specific solutions could be
1. Azure Media Services - https://azure.microsoft.com/en-us/services/media-services/, https://azure.microsoft.com/en-us/services/media-services/encoding/, https://azure.microsoft.com/en-us/services/media-services/live-on-demand/

You will have to figure out pricing part though, best way to do it is to talk to your Account Manager (If you are dealing with Microsoft products, they would have assigned you one).

And for the record, if you already have in-house .net experts, you will be delighted with .Net core's offerings. I was wondering that why not create a single back-end solution which exposes APIs (WebAPI to be more .net specific) for Mobile/Desktop/Web Apps? There are couple of offerings but I am currently looking at :
1. Microsoft PowerApps https://powerapps.microsoft.com/en-us/ - For internal apps
2. Google Flutter https://flutter.dev/ - For external/customer facing mobile/web apps

Note: You have already heard of Xamarin (https://visualstudio.microsoft.com/xamarin/) and if you want to maintain a single base solution then you can have a look at .Net Standard https://docs.microsoft.com/en-us/dotnet/standard/net-standard. This can solve your problem on many front - you can use .Net Standard with
1. .Net / .Net Core / Mono
2. Xmarin.iOS/Xamarin.Mac/Xamarin.Android
3. UWP
4. Unity

Only challenge right now is this is an evolving platform (same goes with Flutter and PowerApps and .Net Core) and empirical evidences are difficult to come by. You will have to take a risk (Or invest some time and energy and resources for a PoC to see if it all works out).

Whew... I think this should open Pandora's box. I will wait for you to process the information (too many links .. I can't help it) and come back.

Regards,
Chinmay.
You got recommendations on how to optimize your video processing on bare metal.
I have a different point of view, and suggest that if possible
  1. Go all into the cloud
  2. Utilize a CDN
First, let me divide the world of video into 2 dimensions, based on how the content is consumed.

VOD vs Live streaming
If your technology and sites are geared at live streaming, then stay with on-premise infrastructure, unless your live streaming is really not sensitive to 30 seconds latency.
If, however, your content is not real time streaming, then go cloud and never look back ( I will explain in a moment).

Bestsellers vs Long tail
If 90% of your traffic will be taken by 10% of your content or less, then you will save bundles of money and improve your overall speed and user satisfaction by using a CDN for your video content. Since RTMP was shut down with the downfall of Flash, modern video content is mostly HTTP compatible and very cacheable.
If, on the other hand, the characterization of your traffic is user-generated-content of grandmothers filming their grandchildren and sharing with some uncles, then maybe your content will a have very low cache hit ratio, and using a CDN will not really help.

About cloud vs on premise
If ALL of the following are true
  • You are happy about the security of your data.
  • You are content with the server's reliability.
  • You are happy with the controls you have regarding data access.
  • The costs of replacing, fixing and upgrading your storage, servers, and equipment are just fine.
  • Your infrastructure will not require major growth soon.
  • You are happy with any monitoring tools (and associated costs in time+licenses).
  • You are content with the availability of your data, and have a fool proof disaster plan.
Then don't switch to the cloud.

But if any of these statements is not true, consider the Cloud alternatives.

Lets look at AWS costs.
If you consider all of the costs involved in running a datacenter, such as cabling, spare parts, remote hands, patch panels, FTE, procurement and legal, and so on, then you will find out that getting your workloads into AWS is comparable to 18 months depreciation.
If you commit to AWS by buying reserved capacity, then over 36 months the costs are the same.
However, with AWS you can start small and do not have to commit. So you pay more per compute unit, but can stop any time, and not have the risk of buying expensive equipment for unsuccessful ventures.

Batch processing like video can be safely engineered to work with Spot instances, and save up to 80% , which is even more than what you would by committing to annual plans.

Some people complain about surprise costs. If you plan it a bit, and maybe buy your cloud compute via a certified reseller, you will have very little surprises coming your way.
Regarding Issue 2.

I am not an expert on cross platform development, Xamarin sounds like a good idea.
However, the bigger problem , in your own words,
How to do updates or new releases that only affect the specified area or feature and ensure these changes don’t break other parts of the website or app?

The solution for that is independent of the development platform, but rather is a fundamental architecture change.
Mainly, switching to SOA, or its new sexy acronym "Micro Services".

As an engineer you can appreciate the beauty of service oriented architecture.
Instead of monolithic applications that do everything in a single code base, you break down the system into separately serviceable, independent components, that interact with each other using predefined, preplanned interfaces.
Is there a bug in the user registration process? Fixing it will not affect the user billing process, because they may even have separate code bases.
Greg PaulInventer / CTO

Author

Commented:
Many excellent comments/suggestions.  Please allow me a day to go through them and comment back to see if I can narrow things down even more.  I will find a way to get points to all the helpful commenters.  

VERY MUCH appreciated.  Be back with you all shortly.

Thanks
Greg PaulInventer / CTO

Author

Commented:
Thanks for all the good ideas up to now.  Some replies to your suggestions and a couple follow up questions.

David Favor

I have not run the numbers recently but I think you may be correct about the cost of cloud server processing video exceeding dedicated server built to encode/transcode.  That is what has kept us on dedicated servers for the video processing portion.  The good thing is our core SaaS technology is a replacement for live video streaming in situations where a customer wants to show updated conditions like roads, etc.  Live streaming is complicated, expensive, requires one camera per stream, doesn’t play on all devices, and the internet couldn’t be a worse platform for transmitting live streams.  It may not be time yet to move our proprietary video software into VM yet but I am more than comfortable using VM for a web server.

Also, unfortunately I am more than aware of AWS surprise invoices.  We have suffered through several and we only using them as a backup location.  That’s another story but a relevant cautionary tale for sure.

About Linux, we have a wealth of ASP.NET programming capabilities.  There are pros and cons to each OS but we don’t have the bandwidth to change OS and do everything else that needs done at this time.

You said, “My guess... if you take the intersection of all these consulting calls together, you may come up with some fantastic design input.”  This very well could be what happens.  Good idea.

John Tsioumpris

Thanks for the confirmation of .NET.  Since I am a novice with cross platforms I have to rely on experienced professionals for guidance.  The .NET CORE seems to be getting a fair amount of thumbs up.

Chinmay Patel

Thanks for taking a crack at what is a complicated question.

Your list of links/options should be very helpful.  Even things like Azure Media could be a good option for the video heavy processing.  But as you said, it is all going to come down to the cost tradeoff.  That includes provisioning new dedicated servers whenever we need to add capacity or availability of our SaaS in different geographies.

Question:  Are you saying that Xamarin for example, will not play on .NET CORE?  If not, would you give up what I am hearing are the major improvements of .NET CORE over ASP.NET?  We don’t have to use Xamarin if there are other equally functional platforms that will help us expedite the app programming and changes.

Also, it is interesting to know that there is little if any empiral evidence for these solutions since as an engineer that is usually very important in making decisions about solutions.

Shalom Carmel

One of the details that I somehow left out is that our video solution is a replacement for live video streaming for certain scenarios where users need a regular, high quality update of conditions of any sort.  We looked at live streaming about ten years ago as a solution and it is NOT a good solution for most situations.  

Excellent checklist “About cloud vs on premise  If ALL of the following are true…”  On the fence on a few of them and exactly why we are looking at cloud solution.  And your break even analysis could have only come from a CTO!  Well done and very helpful.

Question:  You say “Batch processing like video can be safely engineered to work with Spot instances, and save up to 80%”.  Our video processing load is very constant because it is all done on a schedule to provide video updates to different customers.  What is a “Spot instance” and do you think it would still be a savings based on the fact that the workload is constant?

Question2: You say, “buy your cloud compute via a certified reseller”.  How does that stabilize our costs and can you give me an example of a reliable reseller?

Question3:  “The solution for that is independent of the development platform, but rather is a fundamental architecture change. Mainly, switching to SOA, or its new sexy acronym "Micro Services".”.  When I was programming early in my career this is how I did it even though it may not have had a title or acronym.  Naively I thought this is how everyone coded…  Nope.  I didn’t realize it had to be set up as part of the architecture.  But this is EXACTLY what we are moving to even though I didn’t know it had a name.  I call it block programming (BP?).  “As an engineer you can appreciate the beauty of service oriented architecture.” And I thought everyone appreciated the beauty of this.  Live and learn.  What a headache.  Question is this, are there documented guidelines/architecture for this radical programming method or do you just do it?  (Now forever to be called BP)
Greg PaulInventer / CTO

Author

Commented:
Gentlemen - I am not familiar with the generally accepted procedures on this site.  I attempted to get points to everyone who contributed to my complicated scenario.  In case you have not seen my comments, below are my comments back to each of you.

Thanks for the efforts.
Greg



David Favor

I have not run the numbers recently but I think you may be correct about the cost of cloud server processing video exceeding dedicated server built to encode/transcode.  That is what has kept us on dedicated servers for the video processing portion.  The good thing is our core SaaS technology is a replacement for live video streaming in situations where a customer wants to show updated conditions like roads, etc.  Live streaming is complicated, expensive, requires one camera per stream, doesn’t play on all devices, and the internet couldn’t be a worse platform for transmitting live streams.  It may not be time yet to move our proprietary video software into VM yet but I am more than comfortable using VM for a web server.

Also, unfortunately I am more than aware of AWS surprise invoices.  We have suffered through several and we only using them as a backup location.  That’s another story but a relevant cautionary tale for sure.

About Linux, we have a wealth of ASP.NET programming capabilities.  There are pros and cons to each OS but we don’t have the bandwidth to change OS and do everything else that needs done at this time.

You said, “My guess... if you take the intersection of all these consulting calls together, you may come up with some fantastic design input.”  This very well could be what happens.  Good idea.

John Tsioumpris

Thanks for the confirmation of .NET.  Since I am a novice with cross platforms I have to rely on experienced professionals for guidance.  The .NET CORE seems to be getting a fair amount of thumbs up.

Chinmay Patel

Thanks for taking a crack at what is a complicated question.

Your list of links/options should be very helpful.  Even things like Azure Media could be a good option for the video heavy processing.  But as you said, it is all going to come down to the cost tradeoff.  That includes provisioning new dedicated servers whenever we need to add capacity or availability of our SaaS in different geographies.

Question:  Are you saying that Xamarin for example, will not play on .NET CORE?  If not, would you give up what I am hearing are the major improvements of .NET CORE over ASP.NET?  We don’t have to use Xamarin if there are other equally functional platforms that will help us expedite the app programming and changes.

Also, it is interesting to know that there is little if any empiral evidence for these solutions since as an engineer that is usually very important in making decisions about solutions.

Shalom Carmel

One of the details that I somehow left out is that our video solution is a replacement for live video streaming for certain scenarios where users need a regular, high quality update of conditions of any sort.  We looked at live streaming about ten years ago as a solution and it is NOT a good solution for most situations.  

Excellent checklist “About cloud vs on premise  If ALL of the following are true…”  On the fence on a few of them and exactly why we are looking at cloud solution.  And your break even analysis could have only come from a CTO!  Well done and very helpful.

Question:  You say “Batch processing like video can be safely engineered to work with Spot instances, and save up to 80%”.  Our video processing load is very constant because it is all done on a schedule to provide video updates to different customers.  What is a “Spot instance” and do you think it would still be a savings based on the fact that the workload is constant?

Question2: You say, “buy your cloud compute via a certified reseller”.  How does that stabilize our costs and can you give me an example of a reliable reseller?

Question3:  “The solution for that is independent of the development platform, but rather is a fundamental architecture change. Mainly, switching to SOA, or its new sexy acronym "Micro Services".”.  When I was programming early in my career this is how I did it even though it may not have had a title or acronym.  Naively I thought this is how everyone coded…  Nope.  I didn’t realize it had to be set up as part of the architecture.  But this is EXACTLY what we are moving to even though I didn’t know it had a name.  I call it block programming (BP?).  “As an engineer you can appreciate the beauty of service oriented architecture.” And I thought everyone appreciated the beauty of this.  Live and learn.  What a headache.  Question is this, are there documented guidelines/architecture for this radical programming method or do you just do it?  (Now forever to be called BP)
Chinmay PatelChief Technology Ninja
Distinguished Expert 2018

Commented:
Hi Greg,

Just to clarify,
Question:  Are you saying that Xamarin for example, will not play on .NET CORE?  If not, would you give up what I am hearing are the major improvements of .NET CORE over ASP.NET?  We don’t have to use Xamarin if there are other equally functional platforms that will help us expedite the app programming and changes.

Also, it is interesting to know that there is little if any empiral evidence for these solutions since as an engineer that is usually very important in making decisions about solutions.

I am not sure which part of my response gave that impression. I am suggesting that you keep maximum code in .Net Standard Project so it becomes easy to manage across platform. Rest of the stuff, for example, APIs and your web front-end can definitely be built upon .Net Core.
If I were to redesign a system today, I would avoid ASP.Net - You will see over the period of time how things will be in favor of .Net Core. Mobile apps can be in any of the platform, Xamarin will suit your team if they are comfortable with XAML, Visual Studio, C# and .Net.

And evidence is available for .Net Core and Xamarin (but not much on PowerApps, Flutter etc.) as they are relatively mature platforms.

I hope this helps.

Regards,
Chinmay.
David FavorFractional CTO
Distinguished Expert 2018

Commented:
Live video streaming or realtime video changes is fairly easy using PHP pseudo streaming.

Each new 206 range request made can return completely new/generated data.

This approach is extremely lightweight. 1,000,000s of simultaneous streams can run on $100/month machines with low speed connections.
Hey,
Sorry for the delay. Let me add some more information.

Microservices are the current rave, google it to find more. If you call your architecture BP that's fine, as long as it lets you decouple the code base dependencies and gain whatever you want to gain.
 
Spot instances are Amazon's way to sell unused, excess capacity at some cost, rather than let it be wasted for nothing. See here https://aws.amazon.com/ec2/spot/ . Yours is an excellent, classic use case where spot instances can be made to work flawlessly. See spotinst.com for a solution to help you run on spot instances.

When you buy your cloud compute via a certified reseller, that reseller usually has a battle tested methodology to help you lift and shift your workload to the cloud, as well as adapt your workload to the new capabilities. The reseller often will give you custom tools to control and manage your budget and monetary spend, and if you want will sell you ongoing 24/7 support for any level that you need.
Greg PaulInventer / CTO

Author

Commented:
Thanks Shalom.  Very helpful.

Greg

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial