AWS

Amazon Web Services (AWS), is a collection of remote computing services, also called web services, that make up a cloud-computing platform  operated from 11 geographical regions across the world. The most central and well-known of these services include Amazon Elastic Compute Cloud, also known as "EC2", and Amazon Simple Storage Service, also known as "S3". Other services include Elastic MapReduce (EMR), Route 53 (a DNS web service),  provides a highly available and scalable Domain Name System (DNS) web service, Virtual Private Cloud (VPC), storage, database, deployment and application services.

Share tech news, updates, or what's on your mind.

Sign up to Post

how to convert xml to csv in aws by using boto3
0
The eGuide to Automating Firewall Change Control
The eGuide to Automating Firewall Change Control

Today’s IT environment is constantly changing, which affects security policies and firewall rules. Discover tips to help you embrace this change through process improvement & identify areas where automation & actionable intelligence can enhance both security and business agility.

Hi Experts,

       I am running webapplication in aws-ubuntu with nginx connecting uwsgi through sockets.

I connect to this machine through putty ssh. and the ipaddress I use is 54.253.108.37

When I run application,  I get bad gateway errors only for

http://54.253.108.37/iiif

but for http://54.253.108.37/  i get error_connection timeout.

Please see the logs when I run nginx

> ntdl@1.0.2 start-prod /home/ubuntu/bharath/sourcecode/projects/NTDL/ntdl
> concurrently --no-color "uwsgi --emperor code/uwsgi.ini --uid www-data  --gid www-data" "nginx"  "npm run render-prod"

[0] *** Starting uWSGI 2.0.15 (64bit) on [Wed Feb 21 03:31:48 2018] ***
[0] compiled with version: 5.4.0 20160609 on 09 February 2018 02:01:25
[0] os: Linux-4.4.0-1049-aws #58-Ubuntu SMP Fri Jan 12 23:17:09 UTC 2018
[0] nodename: ip-10-244-152-179
[0] machine: x86_64
[0] clock source: unix
[0] pcre jit disabled
[0] detected number of CPU cores: 8
[0] current working directory: /home/ubuntu/bharath/sourcecode/projects/NTDL/ntdl
[0] detected binary path: /usr/local/bin/uwsgi
[0] setgid() to 33
[0] setuid() to 33
[0] *** WARNING: you are running uWSGI without its master process manager ***
[0] your processes number limit is 60090
[0] your memory page size is 4096 bytes
[0] detected max file descriptor number: 1048576
[0] *** starting uWSGI Emperor ***
[1] nginx exited with code 0
[2]
[2] > ntdl@1.0.2 render-prod /home/ubuntu/bharath/sourcecode/projects/NTDL/ntdl
[2] > …
0
I am trying to add a host route as a test to my routing table in AWS to make it go over my DX connection. However for some reason I seem to not be able to get
the route to propagate as can be seen in the attached image. Traceroute fails to make it beyond even the first hop. What can cause this?
No-Prop.png
0
Hi experts,

when I run nginx, I get error,
nginx exited with code 0

nginx is started from npm run scripts

I had run $sudo npm run start-prod
scripts:
start-prod: has the following command

concurrently --no-color "uwsgi --emperor code/uwsgi.ini --uid www-data gid www-data" "nginx"  "npm run render-prod"

npm run render-prod value of script is
 "render-prod": "babel-node node/react-render.jsx"

[0] *** Starting uWSGI 2.0.15 (64bit) on [Thu Feb 15 06:06:34 2018] ***
[0] compiled with version: 5.4.0 20160609 on 09 February 2018 02:01:25
[0] os: Linux-4.4.0-1049-aws #58-Ubuntu SMP Fri Jan 12 23:17:09 UTC 2018
[0] nodename: ip-10-244-152-179
[0] machine: x86_64
[0] clock source: unix
[0] pcre jit disabled
[0] detected number of CPU cores: 8
[0] current working directory: /home/ubuntu/bharath/sourcecode/projects/NTDL/ntdl
[0] detected binary path: /usr/local/bin/uwsgi
[0] setgid() to 33
[0] setuid() to 33
[0] *** WARNING: you are running uWSGI without its master process manager ***
[0] your processes number limit is 60090
[0] your memory page size is 4096 bytes
[0] detected max file descriptor number: 1048576
[0] *** starting uWSGI Emperor ***
[1] nginx exited with code 0
[2]
[2] > ntdl@1.0.2 render-prod /home/ubuntu/bharath/sourcecode/projects/NTDL/ntdl
[2] > babel-node node/react-render.jsx
[2]
[0] *** has_emperor mode detected (fd: 5) ***
[0] [uWSGI] getting INI configuration from code/uwsgi.ini
[0] *** Starting uWSGI 2.0.15 (64bit) on…
0
Hi There,

I am getting the following issue with postgresql in aws ubuntu environment.

When I  run python3 manage.py makemigrations, I get the following error.  
It throws the following error. File "/usr/local/lib/python3.5/dist-packages/psycopg2/init.py", line 164, in connect conn = _connect(dsn, connection_factory=connection_factory, async=async) django.db.utils.OperationalError: could not connect to server: Connection refused Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5432?

Hence I checked whether postgresql is working fine.
I had tried sudo su - postgres It went inside the postgres cmd prompt so it became postgres@ip-10-254-3-58:~$

now when I try psql I get the same error as when I run python3 manage.py makemigrations.

postgres@ip-10-254-3-58:~$psql psql: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

I had tried uninstall and reinstall postgres,

During uninstall , i had tried the following command

$sudo apt-get --purge autoremove postgresql*

I got many errors

Then I had tried
sudo apt-get clean
sudo apt-get update

I got the following errors.  

Hit:1 https://deb.nodesource.com/node_8.x xenial InRelease
0% [1 InRelease gpgv 4,646 B] [Connecting to archive.ubuntu.com (91.189.88.161)] [Connecting to security.ubuntu.com (91.189.91.26)] [Connecting to …
0
Hello,
    I am using the AWS sFTPGateway application made by Thorn Technologies. It lets you connect an EC2 instance to an S3 bucket and then allow sFTP from the outside which then gets stored into the bucket in a folder based on the linux user that you create. When you create a user it puts them in to the sftponly group and they are locked to their home directory, which ends up being /home/user/home/user/*files_go_here. I asked them about using standard FTP and they said while it is not officially supported people have had success by using information found here https://stackoverflow.com/questions/7052875/setting-up-ftp-on-amazon-cloud-server#.

    I went through the steps to install and setup vsftpd, I can hit a login using standard port 21 ftp but it does not seem to authenticate properly and I am not seeing anything in the logs anywhere regarding this. Most of the setup seems the same as the sftpgateway such as having a user created, them being in the sftponly group, etc. The user can be logged in when using port 22, but not port 21. Does anyone have any insight as to what I might need to change to allow this to work? I am not sure if there is something within vsftpd that  I need to change so that it knows each user is now in /home/user/home/user instead of just /home/user, as permissions for that particular user are only on the last /user folder and not on anything before that.

If anyone has any information, I would greatly appreciate  it!

Thanks,
0
I'd like to write a Lambda function that fetches a (1) CloudWatch Metric (that is already being monitored at every 5 minutes), (2) divides that value by 300 seconds and (3) pushes to CloudWatch as a Custom Metric.

The CloudWatch metrics already being monitored are "Volume Read Ops" and "Volume Write Ops" on a could different active EBS volumes.

I have no every on using Java or Python for this.

Thanks for your help or pointing me in the right direction.
0
Hi,

We have an Amazon Echo Dot device.

On our Amazon Alexa account, we have many skills installed.

With these skills, Alexa can connect to various devices such as SmartThings, Philips Hue, etc to control devices which are connected to these hubs.

This Alexa account is a UK based account so we have no accessibility to US based Alexa skills.

We are missing out on connectivity with devices we have and would like to find a way to use both UK and US based Alexa skills.

There is a VPN we have called NordVPN which we have that allows us to connect to other servers for accessing services, applications, etc in other countries.
https://nordvpn.com/

We would like to know:

1. If we could use NordVPN to access and use the US based Alexa skills
2. If we could use NordVPN to access both US and UK based Alexa skills at the same time
3. If there is an easy way to download a US based Alexa skill onto UK based Alexa account and use it on UK account for Alexa
4. Any alternative solution for issue; we would like to download a US based Alexa skill on a UK based Alexa account where we need to also access the UK based skills

Kind regards,
Robbie
0
How do I seamlessly move workload such as my VMwares into AWS EC2s or AWS containers.. and seamlessly move them back into VMware on my data centre premise using any tool out there - will be only using AWS for now.
0
An error occurred (ValidationException) when calling the PutItem operation: One or more parameter values were invalid: Type mismatch for key Xyz expected: S actual: M

Sometimes it throws **expected: S actual:** SS

I am using boto3 for this. I am trying to put an item in DynamoDB table. Below is my python snippet:  

    from boto3.dynamodb.conditions import Key, Attr
    dynamodb = boto3.resource('dynamodb',region_name='us-west-2')
    table = dynamodb.Table('Xyz-Account')
    table.put_item(
        Item={
            'Xyz': {'S': xyz_num},
        })   

Open in new window


xyz_num variable is string type. I cross checked that.And I have checked the table field in the dynamodb, it is a string type field and not number or any other type. Where is the mismatch occurring here ? And by the way, the function ran coolly a while ago, but then some black magic happened and this error came up. I've been trying to resolve this for 3 hours. Can anyone help ? Thanks in advance.
0
Will You Be GDPR Compliant by 5/28/2018?
LVL 1
Will You Be GDPR Compliant by 5/28/2018?

GDPR? That's a regulation for the European Union. But, if you collect data from customers or employees within the EU, then you need to know about GDPR and make sure your organization is compliant by May 2018. Check out our preparation checklist to make sure you're on track today!

If I have an EC2 instance in AWS - say a linux host - and I want it to have Internet access say at
TCP 80, 443 and 22, would a NAT Gateway and a Security group with three rules permitting
those ports be sufficient?
0
Greetings Experts,

I can't seem to figure out how to add the second backup tunnel for AWS on my Sonicwall.  All the directions I'm finding  just show how to create the first tunnel, which I did and the VPN is running.  
In the config file you get two sets of information with different wan IP's.  The Lan Gateways are the same.  
I went to create a second VPN on the Sonicwall with the second AWS gateway IP, but I can't due to the Lan networks being the same.  When I go to create a new network, with the same lan network it says they overlap and won't let me do it.  

I'm guessing I'm just missing something easy, but I'm new to sonicwall and this firmware is new as well.

Any help would be most appreciated.

Kacey
0
AWS - Looking to create an alarm which sends out an email when the root account is used for logging in, there are many articles out there describing what needs to be done but nothing solid to have a solution.

Requirements:
  • Log root account login activity
  • Email notification via SNS

Any help will be appreciated.
0
We are planning to setup AD DC server in AWS. We have two office sites where medium scale firewall present. I Need your assistance how to plan for it. we have around 40 employees in both the location. We need to apply policy for the user who used to work from home. Kindly assist on this.
0
how do you connect to aws sqs locally using java????
0
I am running Windows 10 from within Virtual Box. When I launch Windows PowerShell for AWS and just type "aws" I get the error below.
I get the same error when I on any new AWS Windows 2008-2016 server using their AMI. My Dell laptop runn ing Windows 10 I installed PS for AWS and it worked perfectly.

"aws : The term 'aws' is not recognized as the name of a cmdlet, function, script file, or operable program."
0
I am running SFTP on Ubuntu in AWS.
It is currently setup to direct the users to the specific data folder, and that part works when I assign them to the 'sftp' group.

As I have multiple vendors that will be using this, I need to be able to send each username to specific folder and limit their access to that folder.
How do I do that?
0
Hello Experts,

Can someone please let me know the major benefits of using Hashicorps Terraform instead of Azure ARM?

I know that Terraform is provider agnostic (it can be used with any cloud provider), and I'm aware that its more readable than Azure JSON script.

However, because I no desire to train on AWS, or on Google's Cloud platform, I would like to know what how I would benefit from learning and adopting Terraform if I plan on staying an Azure Cloud solution engineer?

Thanks

Carlton
0
Hello EE,

I need to decommission some azure vms, and move them into AWS. I was following this thread ;

https://awsinsider.net/articles/2017/03/28/migrate-azure-vm-to-aws-1.aspx

However , I do not have access to MS Hyper V manager . I did try an import the vhd using AWS Import-EC2Image without success.  I received
Import-EC2Image : Error making request with Error Code InternalError and Http Status Code InternalServerError. No further error
information was returned by the service.

I suspect that the vhd needs to be sysprepped etc . Is there a suggested work around to this issue ?
0
Free Tool: Site Down Detector
LVL 12
Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

I am trying to install multiple mediawiki farm sharing same resources (4 in total), exactly like Wikisource.org, on Amazon Web Services EC2. This installation will consist of the main wiki in English (wikiexample.org), two languages sub domains (lang1.wikiexample.org, lang2.wikiexample.org) and a commons hosting their media files (commons.wikiexample.org).

The wikis will have the extensions of Wikisource like ProofreadPage, PDF handler, Djvu extension and the Translate extension.

It should be possible to maintain and upgrade the entire wikis centrally and not one at a time.

Can anyone please put me through how to go about this?

Thank you.
0
I powered up our temp server (W2K8R2) to run and schedule some scripts (cmd or bat and maybe ps1). However our backup server (W2K8R2) which issues the snapshots won't run any basic AWS commands although Windows for Powershell for AWS is installed and AWS CLI commands are used for the snapshot. I'm getting the error below. It is running PowerShell 3.0 and I don't want to upgrade it as I am worried that I might break the existing snapshot. I have this same problem with new servers (W2K8R2-W2K16) with no GPO in a workgroup (no custom settings).

=====================================================
PS C:\\Program Files (x86)\\AWS Tools\\PowerShell\\AWSPowerShell> aws aws : The term 'aws' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ aws
+ ~~~
    + CategoryInfo          : ObjectNotFound: (aws:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException  =======================================

Steps taken
1. Added the full to both the system and PowerShell path $env:PSModulePath = $env:PSModulePath + ";C:\\Program Files (x86)\\AWS Tools\\PowerShell\\AWSPowerShell"

2. Ran the Get-Module command, but probably not needed PS C:\\Program Files (x86)\\AWS Tools\\PowerShell\\AWSPowerShell> Get-Module

ModuleType Version    Name                                …
0
Two questions:


1. How do I combine these two scripts? They work by themselves, but I can't combine successfully. I want to ideally get the latest snapshot or any snapshots after 01-01-2018. The current method is grabbing all snapshots which exceed 20,000 and dumps it to a text file, but takes all day to apply the tag. All current snapshots have tags so I just need to tag most recent or anything after 01-01-2018.

aws ec2 describe-snapshots --filters Name=volume-id,Values=vol-abc123 --query Snapshots[*].{ID:SnapshotId} --output text >>c:\\AWS\\snapshots.txt

aws ec2 describe-snapshots --query 'Snapshots[?StartTime >= `2018-01-01`].{id:SnapshotId}'


2. Can we use Lamda or any other AWS service to run the scripts above?
0
I know how to manually tag all snapshots for a given volume by outputting them into a file then pasting that into another file (Excel csv) and running the command (see 2 scripts below). How can I hard code a script to tag all snapshots with a specific key and specific value for a specific volume id or instance ID? Or how I dump all snapshots for a given volume id or instance to a text file and run another script to tag them with a specific key and value (yes it would be repeating 99% every night)? I am looking for a short-term solution. Doesn't have to be neat. Just tag all snapshots for a specific volume id or instance id with a key and value. Now sure how to use an array, but I know what it is.


List all snapshots for a given volume
aws ec2 describe-snapshots --filters Name="volume-id",Values="vol-123456" --query 'Snapshots[*].{ID:SnapshotId,Time:StartTime}' --output text >> C:\Temp\test2.txt

Tag all snapshots with a specific value
aws ec2 create-tags  --filters Name="volume-id",Values="vol-123456"  --query 'Snapshots[*].{ID:SnapshotId,VolumeID:VolumeId}' --tags "Key=Cost,Value=ExpenseAccount01"

Something like this (this is just a made up script)
aws ec2 create-tags --filters Name="volume-id",Values="vol-123456" --tags "Key=Cost,Value=ExpenseAccount01"


I am not sure how to use the scripts below (for loop/array)

:: Create a file with all scheduled snapshots
ec2-describe-snapshots|find /i "%dateDOW%: Daily Backup …
0
Hi,

Looking to dip my feet and have a look at certification

Im a cloud n00b

Thanks
0
I have a PDF document on an Amazon S3 site. I have made it public but when I check the link in an Eventbrite event, access is denied. Any suggestions? Thanks
0

AWS

Amazon Web Services (AWS), is a collection of remote computing services, also called web services, that make up a cloud-computing platform  operated from 11 geographical regions across the world. The most central and well-known of these services include Amazon Elastic Compute Cloud, also known as "EC2", and Amazon Simple Storage Service, also known as "S3". Other services include Elastic MapReduce (EMR), Route 53 (a DNS web service),  provides a highly available and scalable Domain Name System (DNS) web service, Virtual Private Cloud (VPC), storage, database, deployment and application services.

Vendor Experts

Gil FeldmanMonday Learn more about Monday