Spring boot with CORS

CORS (Cross-Origin Resource Sharing) errors occur when a web application running in a browser requests a resource from a different domain or port than the one it originated from. Spring Boot provides built-in support for handling CORS errors. Here are the steps you can follow to solve a CORS error in Spring Boot:

1. Add the following dependencies to your project's build file:

implementation 'org.springframework.boot:spring-boot-starter-web'

2. In your Spring Boot application, create a class that extends the WebMvcConfigurer interface and add the following code to it:

@Configuration public class WebConfig implements WebMvcConfigurer {
    @Override public void addCorsMappings(CorsRegistry registry) {
        registry.addMapping("/**").allowedOrigins("*")
          .allowedMethods("GET", "POST", "PUT", "DELETE", "HEAD")
          .allowedHeaders("*").allowCredentials(true);
    }
}

 This code enables CORS for all endpoints by allowing any origin, HTTP method, and header. It also allows credentials to be passed along with the request.

3. Alternatively, you can enable CORS globally by adding the following configuration to your application.properties file:

spring.mvc.allow-cors=true
spring.mvc.cors.allowed-origins=*
spring.mvc.cors.allowed-methods=GET,POST,PUT,DELETE,HEAD
spring.mvc.cors.allowed-headers=*
spring.mvc.cors.allow-credentials=true

This configuration achieves the same result as the previous step.

By following these steps, you should be able to solve the CORS error in your Spring Boot application.

Happy coding.

Apache HUDI and its Use cases

Apache Hudi is an open-source data management framework that enables high-performance and scalable data ingestion, storage, and processing.

Hudi stands for “Hadoop Upserts Deletes and Incrementals” and is designed to handle large-scale data sets in distributed computing environments such as Apache Hadoop, Apache Spark, and Apache Flink.

One of the main features of Apache Hudi is its support for upserts (updates and inserts), deletes, and incremental data processing. This makes it ideal for use cases where data is constantly changing and requires efficient updates and deletes. Hudi provides a mechanism to efficiently store and process the incremental changes made to the data, making it ideal for use cases such as real-time analytics, stream processing, and machine learning.

Apache HUDI provides a robust and scalable data ingestion framework that can handle a variety of use cases , that make it a valuable data management tool. Here are some examples of use cases where Apache Hudi can be used:

Stream processing: Apache Hudi can be used for stream processing use cases such as real-time analytics, log processing, and event-driven architectures. Hudi provides near real-time data ingestion and processing capabilities, making it suitable for stream processing use cases.

Data warehousing: Apache Hudi can be used for data warehousing use cases such as data integration, data consolidation, and data transformation. Hudi’s support for schema evolution and data versioning makes it ideal for data warehousing use cases.

Data lake management: Apache Hudi can be used for managing data lakes and providing data access control, data lineage, and data governance. Hudi provides a scalable and efficient way to store, manage, and process data in a data lake environment.

Machine learning: Apache Hudi can be used for machine learning use cases such as feature engineering, model training, and model serving. Hudi provides a way to efficiently store and process large datasets, making it suitable for machine learning use cases.

IoT and edge computing: Apache Hudi can be used for IoT and edge computing use cases such as data collection, data processing, and data analysis. Hudi’s support for data partitioning and data compression makes it ideal for handling large amounts of data generated by IoT and edge devices.

Overall, Apache HUDI provides a versatile data management tool that can handle a variety of use cases across different industries and domains.

Here’s an overview of the steps involved in ingesting data into Apache Hudi:

Choose the right storage system: Apache Hudi supports various storage systems such as HDFS, Amazon S3, and Azure Blob Storage. You need to choose the storage system that best suits your needs.

Define the schema: Before ingesting data into Apache Hudi, you need to define the schema for the data. The schema defines the structure of the data, including the field names, data types, and any constraints.

Prepare the data: You need to prepare the data that you want to ingest into Apache Hudi. This includes cleaning, filtering, and transforming the data to match the defined schema.

Configure Hudi options: You need to configure various options such as the storage type, table type, and write operation mode. Hudi provides various write modes such as upsert, insert, and bulk-insert.

Ingest the data: Once you have prepared the data and configured the Hudi options, you can ingest the data into Apache Hudi. You can use various tools such as Apache Spark, Apache Flink, or Apache NiFi to ingest data into Hudi.

Query the data: After ingesting data into Hudi, you can query the data using various tools such as Hive, Presto, or Spark SQL. You can also use Hudi’s own API to query the data.

To Be continued..

Python read file from aws s3

 To read a file from AWS S3 using Python, you will need to use the AWS SDK for Python (Boto3). Here's an example code snippet to read a file from S3:

Preview:

import boto3



# Create an S3 client

s3 = boto3.client('s3')



# Define the bucket and object key

bucket_name = 'my-bucket'

object_key = 'path/to/my/file.txt'



# Read the file from S3

response = s3.get_object(Bucket=bucket_name, Key=object_key)

file_content = response['Body'].read().decode('utf-8')



# Print the file content

print(file_content)

In the code above, you first create an S3 client using the boto3.client method. Then you define the bucket name and object key for the file you want to read. Finally, you use the s3.get_object method to retrieve the file content from S3 and decode it to a string using UTF-8 encoding.

Note that you will need to have the appropriate AWS credentials configured to access the S3 bucket. You can either set these as environment variables or use the AWS CLI to configure them.

java: You aren't using a compiler supported by lombok, so lombok will not work and has been disabled.

 

In order to make projects compile with the existing builds of Lombok processor, as a workaround you can use the flag

-Djps.track.ap.dependencies=false

which should be added to File | Settings | Build, Execution, Deployment | Compiler | Build process VM options field.

This will disable collection of dependencies specified by an annotation processor when Filer methods are called

Deploy Lambda with dependency on AWS

AWS Lambda is a server less compute service that lets you run code without provisioning or managing servers, creating workload-aware cluster scaling logic, maintaining event integrations, or managing runtimes. With Lambda, you can run code for virtually any type of application or backend service - all with zero administration. Just upload your code as a ZIP file or container image, and Lambda automatically and precisely allocates compute execution power and runs your code based on the incoming request or event, for any scale of traffic. You can set up your code to automatically trigger from 140 AWS services or call it directly from any web or mobile app.

Suppose you have written your lambda in python and trying to import you favorite packages which you have used in code then it might create problem for you. eg. like Pandas, numpy 

Although Python AWS lambda give access to all BOTO3 lib resources.

Below are the steps to Import your Lambda code into AWS.

Step 1: Start a Cloud9 Linux instance in AWS

•    Search for Cloud9 in the AWS Services

            

•     Click ‘Create Environment’
•     Name your environment whatever you like (e.g. transformer ) and click next step
•     Keep the environment default settings and click next.

• 

•     Click ‘Create Environment’ and you’re ready to go

Step 2: Creating dependency layer of Pandas library

Enter below steps on the terminal to create package and download dependencies.

mkdir dependency

cd dependency

virtualenv v-env

source ./v-env/bin/activate

pip install pandas

deactivate

Check for python version installed on machine

mkdir python

cd python

cp -r ../v-env/lib64/python3.7/site-packages/* .

cd ..

zip -r panda_layer.zip python

aws lambda publish-layer-version --layer-name pandas --zip-file fileb://panda_layer.zip --compatible-runtimes python3.7

Output after completion of all steps should look like this

Step 3: Adding the Pandas layer to your Lamda function

• Go to the AWS Lambda service and click ‘Create Function’
• Name your function, set the runtime to ‘Python 3.7’, and click ‘Create Function’
• Click on Layers 

• 

• Click on ‘Layers’ in the function designer, then click ‘add a layer’

• Choose custom layers from the layers options and on the dropdown, you should see your Pandas layer. 
• Select that layer and choose your version as 1
• Click ‘Add’.

 You are now good to go with your lambda having dependency of Pandas.

Caused by: java.lang.ClassNotFoundException: javax.xml.bind.JAXBException

Problem :

Caused by: java.lang.ClassNotFoundException: javax.xml.bind.JAXBException

Add these dependencies into your pom file

Solution :

<dependency>
    <groupId>javax.xml.bind</groupId>
    <artifactId>jaxb-api</artifactId>
    <version>2.3.0</version>
</dependency>
<dependency>
    <groupId>com.sun.xml.bind</groupId>
    <artifactId>jaxb-impl</artifactId>
    <version>2.3.0</version>
</dependency>
<dependency>
    <groupId>org.glassfish.jaxb</groupId>
    <artifactId>jaxb-runtime</artifactId>
    <version>2.3.0</version>
</dependency>
<dependency>
    <groupId>javax.activation</groupId>
    <artifactId>activation</artifactId>
    <version>1.1.1</version>
</dependency>

file upload error Resolved [org.springframework.web.HttpRequestMethodNotSupportedException: Request method 'POST' not supported]

Problem :

file upload error Resolved [org.springframework.web.HttpRequestMethodNotSupportedException: Request method 'POST' not supported]



Solution:


If you are using Spring MVC with Spring security then main problem was in spring security.

I resolved this problem by adding ${_csrf.parameterName}=${_csrf.token} 

to end of my form action

HTML:

1 2 3 4

<form method="POST" action="/uploadExcelFile?${_csrf.parameterName}=${_csrf.token}" enctype="multipart/form-data"> <input type="file" name="file" /><br/><br/> <input type="submit" value="Submit" /> </form>