Hey folks,
I am trying to setup a crawler in Glue, but I cannot seem to figure out how to setup the IAM permissions. Can someone help me out?
I borrowed code from https://docs.aws.amazon.com/glue/latest/dg/getting-started-access.html to test this out and I am getting the this error:
An error occurred: MyCrawler2 - Service is unable to assume role arn:aws:iam::############:role/myService-dev-us-east-2-lambdaRole. Please verify role's TrustPolicy (Service: AWSGlue; Status Code: 400; Error Code: InvalidInputException).
Here is the code from my yml:
MyDatabase:
Type: AWS::Glue::Database
Properties:
CatalogId:
Ref: AWS::AccountId
DatabaseInput:
Name: "mycrawler"
Description: "TestDatabaseDescription"
LocationUri: "TestLocationUri"
Parameters:
key1 : "value1"
key2 : "value2"
MyClassifier:
Type: AWS::Glue::Classifier
Properties:
GrokClassifier:
Name: "CrawlerClassifier"
Classification: "wikiData"
GrokPattern: "%{NOTSPACE:language} %{NOTSPACE:page_title} %{NUMBER:hits:long} %{NUMBER:retrieved_size:long}"
MyS3Bucket:
Type: AWS::S3::Bucket
Properties:
BucketName: "crawlertesttarget"
AccessControl: "BucketOwnerFullControl"
MyCrawler2:
Type: AWS::Glue::Crawler
Properties:
Name: "testcrawler1"
DatabaseName:
Ref: MyDatabase
Classifiers:
- Ref: MyClassifier
Targets:
S3Targets:
- Path: arn:aws:s3:::my-dev-consolidate
SchemaChangePolicy:
UpdateBehavior: "UPDATE_IN_DATABASE"
DeleteBehavior: "LOG"
Schedule:
ScheduleExpression: "cron(0/10 * ? * MON-FRI *)"
Role:
"Fn::GetAtt":
- IamRoleLambdaExecution
- Arn