Link to home
Start Free TrialLog in
Avatar of gp
gpFlag for United States of America

asked on

Kafka micro service Jenkins OpenShift configuration related issues

Hi,

I have kafka micro service with application.yml  like below in intellij IDEA for springboot 3.0.5 and JDK 17 application 


#ACTIVE PROFILE
spring:
  profiles.active: ${ENV}
  profiles.include: stored-procedure
---
spring:
  aaa:
    props:
      abc-exception-code-indicator: BB,CC,DD
      abc-process--src-code: EE,FF
---
spring:
  config:
    activate:
      on-profile: "local"
  abcSwitch: true
  abcDataSwitch: true
  datasource:
    master-jndi-name: AAAMASTERDB
  kafka:
    template:
      default-topic: kafkas-topic
  properties:
      #security.protocol: SSL
      security.protocol: PLAINTEXT
    consumer:
      bootstrap-servers: localhost:9092
       
---
spring:
  config:
    activate:
      on-profile: "dev"
  abcSwitch: true
  abcDataSwitch: true
  datasource:
    master-jndi-name: AAAMASTERDB
  kafka:
    template:
      default-topic: kafkas-topic
  properties:
      #security.protocol: SSL
      security.protocol: PLAINTEXT
    consumer:
      bootstrap-servers: localhost:9092    
       
---
spring:
  config:
    activate:
      on-profile: "staging"
  abcSwitch: true
  abcDataSwitch: true
  datasource:
    master-jndi-name: AAAMASTERDB
  kafka:
    template:
      default-topic: kafkas-topic
  properties:
      #security.protocol: SSL
      security.protocol: PLAINTEXT
    consumer:
      bootstrap-servers: localhost:9092    
       
---
spring:
  config:
    activate:
      on-profile: "production"
  abcSwitch: true
  abcDatafeedSwitch: true
  datasource:
    master-jndi-name: AAAMASTERDB
  kafka:
    template:
      default-topic: kafkas-topic
properties:
      #security.protocol: SSL
      security.protocol: PLAINTEXT
    consumer:
      bootstrap-servers: localhost:9092    



When I do Jenkins build then build is successful. But while building, deploying image on OpenShift 4  I am getting below type of errors. 

Caused by: java.lang.IllegalArgumentException: Could not resolve placeholder 'spring.datasource.master-jndi-name' in value "${spring.datasource.master-jndi-name}"
    at org.springframework.util.PropertyPlaceholderHelper.parseStringValue(PropertyPlaceholderHelper.java:180)
    at org.springframework.util.PropertyPlaceholderHelper.replacePlaceholders(PropertyPlaceholderHelper.java:126)
    at org.springframework.core.env.AbstractPropertyResolver.doResolvePlaceholders(AbstractPropertyResolver.java:239)
    at org.springframework.core.env.AbstractPropertyResolver.resolveRequiredPlaceholders(AbstractPropertyResolver.java:210)
    at org.springframework.context.support.PropertySourcesPlaceholderConfigurer.lambda$processProperties$0(PropertySourcesPlaceholderConfigurer.java:191)
    at org.springframework.beans.factory.support.AbstractBeanFactory.resolveEmbeddedValue(AbstractBeanFactory.java:920)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1358)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1337)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.resolveFieldValue(AutowiredAnnotationBeanPostProcessor.java:710)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:693)
    at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:133)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessProperties(AutowiredAnnotationBeanPostProcessor.java:482)



If I change ${ENV} value with hard coded environment profile value like "local" I get Kafka boot strap server localhost:9092 not recognized error.

If I change below line (I am running zookeeper server, Kafka server on my local windows laptop only by starting from command prompt)
bootstrap-servers: localhost:9092

as below by relacing localhost with machine name
bootstrap-servers: WABC999F1111.us.aaa.com:9092

I am getting errors like below 

Error starting ApplicationContext. To display the condition evaluation report re-run your application with 'debug' enabled.
15:40:19.512 [main] ERROR o.s.boot.SpringApplication - Application run failed
org.springframework.context.ApplicationContextException: Failed to start bean 'org.springframework.kafka.config.internalKafkaListenerEndpointRegistry'
 
Caused by: org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:830)
 
Caused by: org.apache.kafka.common.config.ConfigException: No resolvable bootstrap urls given in bootstrap.servers
    at org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses(ClientUtils.java:89)
    at org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses(ClientUtils.java:48)
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:732)

How to declare {ENV} profile properly in YAML file to load corresponding required environment? How to define Kafka localhost:9092 so that it is recognized by OpenShift 4.0 without any issues. Please advise.

ASKER CERTIFIED SOLUTION
Avatar of dfke
dfke

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of gp

ASKER

Below bolded, underlined highlighted declaration does not seem to work

# ACTIVE PROFILE
spring:
  profiles.active: ${ENV}

Is it {{ENV}} I have to use like mentioned somewhere in below document


https://docs.openshift.com/container-platform/3.11/dev_guide/templates.html#writing-parameterssue now though.