Dropbox Api + Node.Js를 사용하여 이미지 호스팅 사이트 만들기 (작성중~)

학교 프로젝트로 라즈베리 파이에서 찍힌 사진을 web 서버에 전송하고 사이트를 통해 보여줘야하는 상황이 생겼따.

근데 막상 사진을 전송하는 코드들을 찾아보니 그렇게 썩 맘에 들지도 않았고 무엇보다도

사진을 내 db에 저장해야하는게 조금 무리였달까낫...

그래서 방법을 이러쿵 저러쿵 찾아보니 dropbox에서 api를 지원해줘서

upload, download, image hosting까지 지원해준 다는 것이다!

dropbox api를 사용한다면 라즈베리파에이 찍힌 사진을 내 dropbox에 upload만 하고

내 web 서버는 그 api를 통해 사진만 쫘라락 긁어오면 되니 얼마나 편할까-

라는 생각이 들어서 여러가지 방법을 찾아보았는데, 나에겐 여러 결격 사항들이 있어서 좀 힘들어따.

1. javascripts를 유연하게 사용하지 못함.. 실질적 개발 경험은 node js + ejs + express +heroku 개발 딱 한번 뿐

2. 영어를 못하는데 dropbox api에 대한 정보가 한국에 많이 없음

3. 글을 차근차근 못읽음

솔직히 1번과 2번은 어떻게 어떻게 해결할 수 있는데 3번의 결격사항 때문에

1번과 2번을 해결하는 과정이 너무 힘들었다ㅠㅠ

그렇게 dropbox api 가이드 사이트 들락날락을 한 20번정도 하던 찰나에 dropbox api 가이드 첫장에서 여러 예제들을 공유하는 것을 볼 수 있었다!

https://www.dropbox.com/developers/reference

심지어 언어별로 예제도 제각각이었는데 내가 그나마 조금 할줄아는 javascripts에 심지어 node.js에! 심지어 heroku로! 갤러리를 만들어보자니!! 완전 대박이자낭ㅎㅎ

그래서 이번 글을 쓰게된 이유가 나왔답.

https://github.com/dropbox/nodegallerytutorial?_tk=guides_lp&_ad=javascript_tutorial1&_camp=photo

dropbox/nodegallerytutorial

(https://github.com/dropbox/nodegallerytutorial)

이렇게 깃허브를 통히 친히 튜토리얼까지 만들어가주었기 때문에 너무나도 고맙다고 생각한다.

난 이 깃허브 튜토리얼을 그대로 따라할것이기 때문에 내 능력껏 개발해서 쓰는 글은 아니지만,

저런 3가지 결격사항을 갖고있음에도 따라하는 사람도 있으니 함께 해보자 라는 취지에서 글을 써본당ㅎㅎ

사진이나 뭐 설명 세부적인건 생략하고 코드로 간단간단하게 진행하는 과정을 풀어보겠다!

1. Node.JS와 Express

일단 node js를 깔고 온다! 난 깔려있어서 패스!

그리고 express cli를 설치해준다

npm install express-generator -g

express 커맨드를 통해 간단한 익스프레스 홈페이지 + hbs(handlebars 템플릿 엔진)를 생성할 수 있다. 처음 써본당!

express --hbs cattus

근데 난 이 부분이 되질 않았다!

처음에는 환경변수 등록 문제인것 같아 환경변수를 등록했었다.

(환경변수 등록 방법 : 시스템 환경변수 편집 -> 환경변수 -> 사용자변수 새로 만들기 -> 변수 이름 : (아무거나) 경로 : npm이 설치 되어있는 경로 ex)";C;\Users\사용자이름\AppData\Roaming\npm;"; )

그런데도 여전히 안되길래 구글링을 찾아보다가 나는 powershell을 사용하는데 모두 cmd를 사용하는 것 같아서 cmd로 커맨드를 치니 인식이 되었다!

아 그리고 나는 dbximg라는 이름으로 생성하지 않고 내 학교 프로젝트 이름으로 생성했다.

4. 아마 생성하면 깃허브의 사진처럼

이렇게 파일이 만들어졌을 것이다.

일단 내 딴엔 이렇게 해석했다ㅠㅠ(오역 수정 환영이요!!!ㅠㅠ)

• package.jsonlists the dependencies to be installed and general info on the project. Also declares the entry point of the app, which is the bin/www file.
  프로젝트에 설치된 패키지와 프로젝트 정보를 표시한다. 또한 이 프로젝트의 시작위치를 선언한다. 이 파일은 bin/www file로 표기되어있는듯..??
• bin/wwwis the file that declares the port and actually gets the server running. You won’t need to modify it in this sample.
  bin/www는 포트를 선언하고 서버를 실행하는 파일이다. 우리는 이 예제에서 이 파일을 수정할 필요가 없다고 함!
• app.jsmiddleware libraries get initialized on this file. Think about middleware libraries like code that gets raw texts requests from the Web, transforms them and then formats them nicely.
  이 파일에서 미들웨어라이브러리가 초기화된다고한다..! 가공하지 않은 텍스트 요청을 변환하고 형식화한다고 한다......
• routes/index.jsthis code gets executed when you hit an endpoint on the server.
  이 코드는 마지막에 실행된다는 뜻인듯.
• public folder:is the front end or resources and files that will eventually find their way to the user. Never store any critical code here, as it could be accessed by users via their browser, that is what the back end is for.
  퍼블릭 폴더는 프론트엔드나 파일의 자료로 사용자에게 전달된다. 중요한 백엔드 코드들은 여기다가 저장하지 마라는 뜻인듯

영알못이라 그런지 해석도 한나절이 걸리는듯하당

뭐 일단, 이렇게 대충 읽어두고 앞으로 우리가 사용할 파일들이라는 뜻이다!

이제 우리가 생성한 프로젝트 파일 (여기선 dbximgs)폴더로 이동해서 npm을 설치해 주자!

아마 이미 깔아놔서 그런것일진 모르겠지만... 파일이 두개의 취약성 파일이 발견됐다길래 난 일단 저 친구가 치라는 대로 npm audit fix를 통해 파일을 고쳐주었다.

npm start

일단 npm start를 하면 local 서버가 실행되면서 간단한 사이트가 나오게된다!

서버를 종료하려면 ctrl + C 를 누르장

심플행

2. Front end and back end

원본에서는 갤러리아라고 불리는 자바스크립트 라이브러리를 사용할 것이라고 되어있다. 근데 이 링크를 타고 들어가면 링크가 없어져있다ㅠㅠ (https://galleria.io/)

물론 깃허브에 올려진 코드를 통채로 받아서 사용하는 방법도 있따. 하지만 그것보단 그래도 갤러리아라는 이 프레임워크 최-신 버전을 응용해보면서 공부하는것도 나쁘지 않을 것 같아서 현재 갤러리아에서 공식 배포하는 1.6.1 버전을 받는다.

깃허브 원본 사진과는 정말 구성이 많이 달라보인다

대충 폴더를 스-윽 훝어보면 예전버전과 구성이 많~이 다르진 않다! 대충 dist 파일과 src 파일이 있는데, 처음엔 이거 둘이 뭐가 다른거지 하고 그냥 폴더 내용물을 들여다 보았다.

하지만 구글링을 통해 dist는 실행된 결과물을 저장하는 파일이고 src는 프론트엔드의 원 소스??이런 느낌인것 같다. 그래서 결국 고른 폴더로 dist 폴더를 골랐다.... min.js도 있고 무엇보다 src보다 파일 용량이 훨씬 더 크길래... ㅠㅠ야매식 판단법..

예쁘게 이름도 바꿔서 모셔 놓은 모습

Our first exercise is to put a simple template page running using handlebars. If you want to know more about how to use handlebars,here is a good resource.

"우리의 첫번째 활동은 handlebars를 이용하여 작동하는 간단한 템플릿 페이지를 만드는 것입니다. handlebars에 대해 더 알고 싶다면 여기 좋은 자료가 있습니다"

A template generates HTML code on the fly before it gets sent to the client. Our first exercise will receive a request on the home endpoint/, package the path of several images on an array and pass it to a handlebars template.

템플릿은 클라이언트에게 보내지기 전에, 즉석에서 HTML코드를 생성합니다. 우리의 첫번째 활동은 home endpont / 에서 요청을 받고, array에 있는 여러개의 이미지의 경로를 패키징하여 handlebars템플릿으로 전달할것입니다.

1. First put any three images in the/public/imagesfolder, you can call thema*.jpg*,b*.jpg* andc*.jpg* for simplicity.
   먼저 /public/images 폴더에 3개의 아무 이미지를 넣으세요. 간단하게 a,b,c.jpc라고 부를것입니다.

2. Add the route: in theroutes/index.jsfile, replace therouter.getmethod with the following code.
   route 추가하기 : routes/index.js에서 router.get메소드를 이 코드로 바꿔버리세요.

routes/index.js

//first add the reference to the controller
var controller = require('../controller');

/* GET home page. */
router.get('/', controller.home);

내 프로젝트는 고양이가 주제기 때문에 귀여운 고양이 a.jpg b.jpg c.jpg를 모셔와보았다

2번은 아마 원래(좌) 이 코드를 이렇게(우)로 바꾸라는 뜻인듯?

3. Add the controller: add the implementation of the endpoint in the controller. If you haven’t, create acontroller.jsfile at the root level and add the following code.

컨트롤러 추가하기 : controller의 마지막 부분에 코드를 추기하세요. 만약 이 파일이 없다면 직접 controller.js를 파일의 최상단에 생성하고 이 코드를 추가하세요

controller.js

module.exports.home = (req,res,next)=>{
  var paths = ['images/a.jpg','images/b.jpg','images/c.jpg'];              
  res.render('gallery', { imgs: paths, layout:false});
};

나는 controller.js파일이 없었기 때문에 새로 추가해 주었다.

Theres.rendergets two arguments: the first is the name of the template (which we have not created yet) and the second is a JSON object that will be passed to the template engine. In this case, we will pass our paths array as imgs and the layout:false will ensure that handlebars doesn’t use the template layout.

Now create the template /views/gallery.hbs and copy this code

res.reder은 두개의 arguments값을 가집니다. 첫번째는 템플릿의 이름(아직 안만듬). 두번째는 템플릿 엔진에 전달될 json오브젝트입니다. 우리는 이미지경로 array와 layout:false를 전달할 것입니다.

 

ㅠㅠ... 아래부터 redis까지 해석한 부분이 다 날아가버렸다.. 눈물...

 

/views/gallery.hbs

<!DOCTYPE html>
<html>
<head>                       
      <script type='text/javascript' src='https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js'></script>
      <script src="/galleria/galleria-1.5.7.min.js"></script>
      <script type='text/javascript' src='/javascripts/page.js'></script>
      <link rel="stylesheet" href="/stylesheets/page.css">
</head>
<body>
      <div class="galleria">
      {% raw %}
        {{#each imgs}}
            <img src="{{this}}">
        {{/each}}
      {% endraw %}
      </div>
</body>
</html>

You can see in the body part that we iterate through theimgsobject passed creating HTML code with an image tag per array element. Let’s now add the css and JavaScript file referenced in the header above.

 

public/javascripts/page.js

 jQuery(document).ready(function(){
      Galleria.loadTheme('/galleria/themes/classic/galleria.classic.min.js');
      Galleria.run('.galleria');
});

public/stylesheets/page.css

.galleria{ max-width: 100%; height: 700px; margin: 0 auto; }

🎯The source code at this point can be found inthis link

Now let us run the server with

$npm start

And in your browser navigate tohttp://localhost:3000and it will look like this

Now that we have our front end running. Let us do the back end part, which is the main focus of this tutorial.

3. Dropbox app

We want to access the Dropbox folder of a user who authorizes the middleware to read it and populate a gallery.

To do this, we will first need to create a Dropbox app. For that you need a Dropbox account. If you don’t have one, create one first and then go tohttps://www.dropbox.com/developers/appsAnd click onCreate App

Then choose Dropbox API, App Folder, put a name on your app and click onCreate App.

We choseApp folderpermission so the middleware can only read and write to a single folder to those users who authorize the app.

After this, you want to also enable additional users in this app, otherwise only you can use it. In the settings page of your app you will find a button to do this.

4. OAuth with authorization code grant flow

This application should be able to read a specific app folder for any Dropbox user who consents. For this we need to build an authentication flow where user is redirected to Dropbox to enter credentials and then authorize this app to read users Dropbox. After this is done, a folder inside Dropbox will be created with the name of this app and the middleware will be able to access the contents of that folder only.

 

The most secure way to do this is using an authorization code flow.In this flow, after the authorization step, Dropbox issues a code to the middleware that is exchanged for a token. The middleware stores the token and is never visible to the Web browser. To know who is the user requesting the token, we use a session. At first, we will simply use a hardcoded session value and save it in a cache, but later we will replace it with a proper library to manage sessions and cookies and will be stored on a persistent database.

 

Before writing any code, we need to do an important configuration step in Dropbox:

 

Pre-register a redirect URL in the Dropbox admin console. Temporarily we will use a localhost endpoint which is the only permitted http URL. For anything different to home, you need to use https. We will use a/oauthredirectendpoint. So enter the URL http://localhost:3000/oauthredirect and press the Add button.

Also we we will not use implicit grant, so you can disable it.

 

💡If you are interested in learning more about OAuth, this is agood read👍

The whole authorization flow will have all the following steps which I will explain right after.

Let us write all the code now…👨‍💻

 

First we need a number of configuration items in the config.js file at the root level. You will need to replace the appkey/secret from your own Dropbox console.

config.js

module.exports = {
  DBX_API_DOMAIN: 'https://api.dropboxapi.com',
  DBX_OAUTH_DOMAIN: 'https://www.dropbox.com',
  DBX_OAUTH_PATH: '/oauth2/authorize',
  DBX_TOKEN_PATH: '/oauth2/token',
  DBX_APP_KEY:'<appkey_in_dropbox_console>',
  DBX_APP_SECRET:'<appsecret_in_dropbox_console>', 
  OAUTH_REDIRECT_URL:"http://localhost:3000/oauthredirect",
}

🛑⚠️If you are using a version control system such as git at this point, remember the Dropbox key/secret will be hard coded in some version of your code, which is especially bad if you are storing it on a public repository. If that is the case, consider usingdotenvlibrary along with the .gitignorefile explained on section 7.

Now let’s add the business logic. To create a random state we will use the crypto library (which is part of Node) and to temporarily store it in a cache, we will use node-cache library. The node-cache simply receives a key/value pair and an expire it in a number of seconds. We will arbitrarily set it to 10 mins = 600 seconds.

Let us first install the node-cache library

$npm install node-cache --save

💡The --save adds a dependency in the package.json file.

For the steps 1,2 and 3 in the flow above, modify thehomemethod in thecontroller.jsIf there is no token, we redirect to the/loginendpoint passing a temporary session in the query. Remember we will change this for a session library later.

controller.js

const 
crypto = require('crypto'),
config = require('./config'),
NodeCache = require( "node-cache" );
var mycache = new NodeCache();

//steps 1,2,3
module.exports.home = (req,res,next)=>{    
    let token = mycache.get("aTempTokenKey");
    if(token){
        let paths = ['images/a.jpg','images/b.jpg','images/c.jpg'];              
        res.render('gallery', { imgs: paths });
    }else{
        res.redirect('/login');
    }
}

//steps 4,5,6
module.exports.login = (req,res,next)=>{

    //create a random state value
    let state = crypto.randomBytes(16).toString('hex');

    //Save state and temporarysession for 10 mins
    mycache.set(state, "aTempSessionValue", 600);

    let dbxRedirect= config.DBX_OAUTH_DOMAIN 
            + config.DBX_OAUTH_PATH 
            + "?response_type=code&client_id="+config.DBX_APP_KEY
            + "&redirect_uri="+config.OAUTH_REDIRECT_URL 
            + "&state="+state;

    res.redirect(dbxRedirect);
}

Now we need to list the login endpoint in theroutes/index.js, so add the following line.

routes/index.js

router.get('/login', controller.login);

At this point, you can test it again by running

$npm start

and hittinghttp://localhost:3000should forward to an authentication/authorization page like this

Once you authorize, you will see an error as we have not added an endpoint to be redirected back, but take a look at the url, you will see there the state you sent and the code from Dropbox that you will use to get a token .

Exchanging code for token

When Dropbox redirects to the middleware, there are two possible outcomes:

• A successful call will include a code and a state
• An error call will include an error_description query parameter

In the success case, we will exchange the code by a token via a POST call to the /oauth2/token call in Dropbox. To make that call we will use the request-promise library, which is a wrapper to the request library adding promise capabilities on top of it.

Let us first install the request-promise and request libraries with the following command

npm install request request-promise --save

Now add one more method to the controller with the logic to exchange the code via the Dropbox API. Once the token is obtained we will temporarily save it on cache and redirect to the home path.

 

controller.js

//add to the variable definition section on the top
rp = require('request-promise');

//steps 8-12
module.exports.oauthredirect = async (req,res,next)=>{

  if(req.query.error_description){
    return next( new Error(req.query.error_description));
  } 

  let state= req.query.state;
  if(!mycache.get(state)){
    return next(new Error("session expired or invalid state"));
  } 

  //Exchange code for token
  if(req.query.code ){

    let options={
      url: config.DBX_API_DOMAIN + config.DBX_TOKEN_PATH, 
          //build query string
      qs: {'code': req.query.code, 
      'grant_type': 'authorization_code', 
      'client_id': config.DBX_APP_KEY, 
      'client_secret':config.DBX_APP_SECRET,
      'redirect_uri':config.OAUTH_REDIRECT_URL}, 
      method: 'POST',
      json: true }

    try{

      let response = await rp(options);

      //we will replace later cache with a proper storage
      mycache.set("aTempTokenKey", response.access_token, 3600);
      res.redirect("/");

    }catch(error){
      return next(new Error('error getting token. '+error.message));
    }        
  }
}

The beauty of using the request-promise library and ES7 async await is that we can write our code as if it was all synchronous while this code will not actually block the server. The await indicator will simply yield until therp(options)call has a returned a value (or error) and then it will be picked up again. Notice that the function has to be marked async for this to work. If the promise fails, it will be captured by the catch and we pass it to the app to handle it, so it is pretty safe.

💡If you have any questions on how the options for the request are formed, you can check

the request documentation.💡If you wan to know more about async await this is agood source

Now we need to hook the route to the controller in the routes/index.js file.

 

routes/index.js

router.get('/oauthredirect',controller.oauthredirect);

and run the server again withnpm startand try again http://localhost:3000 You should see again the gallery with the mock images displaying correctly.

5. Fetching images from Dropbox

Now that we are able to see a gallery of images. We want to read the images from Dropbox. After the user authorizes the application to read a folder in Dropbox, a folder will be created within theAppsfolder with the name of this app, in this case dbximgs demo. If theAppsfolder didn’t exist before, it will be created. So go ahead and populate that folder with some images you want. For security purposes we will use temporary links that are valid only for 4 hours.

Now we need to make a call to the Dropbox API to fetch temporary links for those images. We will follow these steps:

1. Call Dropbox/list_folderendpoint which returns information about the files contained in the App/dbximgs demo
2. Filter the response to images only, ignoring other types of files and folders
3. Grab only thepath_lowerfield from those results
4. For eachpath_lowercall theget_temporary_linkendpoint, this link is valid for 4 hours.
5. Grab thelinkfield of the response
6. Pass all the temporary links to the gallery

💡More information about these endpoints in theDropbox documentation

First, you need to add a couple configuration fields

 

config.js

DBX_LIST_FOLDER_PATH:'/2/files/list_folder',
DBX_LIST_FOLDER_CONTINUE_PATH:'/2/files/list_folder/continue',
DBX_GET_TEMPORARY_LINK_PATH:'/2/files/get_temporary_link',

This is the code you need to add to controllers.js

controller.js

/*Gets temporary links for a set of files in the root folder of the app
It is a two step process:
1.  Get a list of all the paths of files in the folder
2.  Fetch a temporary link for each file in the folder */
async function getLinksAsync(token){

  //List images from the root of the app folder
  let result= await listImagePathsAsync(token,'');

  //Get a temporary link for each of those paths returned
  let temporaryLinkResults= await getTemporaryLinksForPathsAsync(token,result.paths);

  //Construct a new array only with the link field
  var temporaryLinks = temporaryLinkResults.map(function (entry) {
    return entry.link;
  });

  return temporaryLinks;
}


/*
Returns an object containing an array with the path_lower of each 
image file and if more files a cursor to continue */
async function listImagePathsAsync(token,path){

  let options={
    url: config.DBX_API_DOMAIN + config.DBX_LIST_FOLDER_PATH, 
    headers:{"Authorization":"Bearer "+token},
    method: 'POST',
    json: true ,
    body: {"path":path}
  }

  try{
    //Make request to Dropbox to get list of files
    let result = await rp(options);

    //Filter response to images only
    let entriesFiltered= result.entries.filter(function(entry){
      return entry.path_lower.search(/\.(gif|jpg|jpeg|tiff|png)$/i) > -1;
    });        

    //Get an array from the entries with only the path_lower fields
    var paths = entriesFiltered.map(function (entry) {
      return entry.path_lower;
    });

    //return a cursor only if there are more files in the current folder
    let response= {};
    response.paths= paths;
    if(result.hasmore) response.cursor= result.cursor;        
    return response;

  }catch(error){
    return next(new Error('error listing folder. '+error.message));
  }        
} 


//Returns an array with temporary links from an array with file paths
function getTemporaryLinksForPathsAsync(token,paths){

  var promises = [];
  let options={
    url: config.DBX_API_DOMAIN + config.DBX_GET_TEMPORARY_LINK_PATH, 
    headers:{"Authorization":"Bearer "+token},
    method: 'POST',
    json: true
  }

  //Create a promise for each path and push it to an array of promises
  paths.forEach((path_lower)=>{
    options.body = {"path":path_lower};
    promises.push(rp(options));
  });

  //returns a promise that fullfills once all the promises in the array complete or one fails
  return Promise.all(promises);
}

Finally, modify again the home method in the controller.js to look like the code below. First of all, you will notice we added an async modifier as we use an await call to get the links from Dropbox from the code above.

 

controller.js.home method

//steps 1,2,3
module.exports.home = async (req,res,next)=>{    
  let token = mycache.get("aTempTokenKey");
  if(token){
    try{
      let paths = await getLinksAsync(token); 
      res.render('gallery', { imgs: paths, layout:false});
    }catch(error){
      return next(new Error("Error getting images from Dropbox"));
    }
  }else{
  res.redirect('/login');
  }
}       

You can run the server and test it. You should be able to see the images from the folder in your gallery after login into Dropbox.

 

👁️Make sure you have images in the folder created after you login to Dropbox and authorize the application.

🎯The source code at this point can be found in this link

6. Cookies, sessions and Redis database

Until now, we use a hardcoded session in the/loginendpoint. We are going to make several changes for the sake of security and it comes with three Web dev components: cookies, sessions and a session store.

Cookies:data stored as plain text on the users browser. In our case it will be a sessionID.Session: set of data that contains current status of a user as well as a token to access Dropbox resources. Identified via sessionID.Session store:where sessions are stored. We use Redis as this is a fast, lean and popular key value storage.

Our new flow will be something like this:

1. When a user hits our main server endpoint/for the first time, a session gets automatically created and a sessionID gets stored via cookies in the browser. We check if the session has a current token.
2. If there is no token, we redirect to theloginendpoint, we create a random state value and store the sessionID in cache with the state as key.
3. When we get redirected back, we find in the cache a SessionID for that state and compare against the current sessionID. This indicates we originated that authentication flow.
4. When the OAuth flow is complete, we regenerate the session (creating a new sessionID) and store the token as part of the new session.
5. We redirect back to our main endpoint/.
6. As a token is found in the current session, the gallery data is returned using the token.

Installing redisTo test this locally, you need to install Redis in your machine which can be obtainedhere

Once you unpack redis on your local machine, just go to the redis folder and run.

$src/redis-server

You don’t need to worry about configuration as this is only a local test instance, the production one will be using Heroku. When it runs, it will look like this:

We will also need the following Node libraries

express-session:Node library to manage sessionsexpress-sessions:Node library that wraps the session storeredis:Node library to manipulate redis

so run the following commands:

$npm install express-sessions express-session redis --save

You need to add a sessionID secret in the config file (This is the secret used to sign the session ID cookie). Pick your own.

config.js

SESSION_ID_SECRET:'cAt2-D0g-cOW',

And now initialize the libraries in the app.js file where any middleware gets configured with the following configuration. Simply add the code below right after the initialization of theappvariable.var app = express();

app.js

var config = require('./config');
var redis = require('redis');
var client = redis.createClient();
var crypto = require('crypto');
var session = require('express-session');

//initialize session
var sess = {
    secret: config.SESSION_ID_SECRET,
    cookie: {}, //add empty cookie to the session by default
    resave: false,
    saveUninitialized: true,
    genid: (req) => {
            return crypto.randomBytes(16).toString('hex');;
          },
    store: new (require('express-sessions'))({
        storage: 'redis',
        instance: client, // optional 
        collection: 'sessions' // optional 
    })
}
app.use(session(sess));

Finally, we will make 5 changes in the controller: 1 in the login method, 1 in the home method, 2 in the oauthredirect method and we will also add a new method to regenerate a session.

1. We save now in the cache the sessionID instead of a hardcoded value.

controller.js login method

// mycache.set(state, "aTempSessionValue", 600); 
mycache.set(state, req.sessionID, 600);

1. Instead of reading the token from cache, read it from the session.

controller.js home method

//let token = mycache.get("aTempTokenKey"); 
let token = req.session.token;

1. In the oauthredirect, now we actually make sure that the state value we have just received from Dropbox is the same we previously stored.

controller.jsoauthredirectmethod

//if(!mycache.get(state)){ 
if(mycache.get(state)!=req.sessionID){

1. For security reasons, whenever we get a new token, we regenerate the session and then we save it. Let us use a method called regenerateSessionAsync that receives the request.

controller.jsoauthredirectmethod

//mycache.set("aTempTokenKey", response.access_token, 3600); 
await regenerateSessionAsync(req); 
req.session.token = response.access_token;

1. Now we implement the regenerateSessionAsync method. This method simply wraps the generation of the session in a Promise. We do this because we don’t want to mix awaits and callbacks. If we had to do this more often we would use a wrapping library, but this is the only time, so we do it in the rough way.💡you can read more about asynchronous callshere

controller.jsregenerateSessionAsyncmethod

//Returns a promise that fulfills when a new session is created
function regenerateSessionAsync(req){
  return new Promise((resolve,reject)=>{
    req.session.regenerate((err)=>{
      err ? reject(err) : resolve();
    });
  });
}

And you can now run withnpm start.🎯The source code at this point can be found inthis link

7. Deploying to Heroku

⚠️While you will not be charged anything for following any of the steps below, provisioning the Redis database addin requires you to have a credit card on file on Heroku. But again,**we will be using only free tiers.**

• First create a free account athttps://heroku.com
• If they ask you about the technology you will be using, select Node.JS
• Then click onCreate an Appand give it a name. In this specific case, I will call itdbximgsas I was lucky enough to have that name still available.

Once you create the app, you are pretty much given the instructions to put the code there,but don’t do all the stepsjust yet as you need to make several changes.

1. Install theHeroku CLIand login

   $heroku login

2. A the root level of your project, initialize a git repository. Make sure you use your own project name. git init heroku git:remote -a dbximgs

3. Now we need to add a .gitignore file at the root level so we don’t upload to Heroku all the libraries or files we want to leave behind for security purposes.

.gitignore

# Node build artifacts
node_modules
npm-debug.log

# Local development
*.env
package-lock.json

4. There is information that should not be hardcoded in the source code like the Dropbox client/secret and also the Redis secret. Also, there are items that should be configured for a local test vs server production such as the redirectURL of the OAuth flow. There are several ways to work on this, like theHeroku localcommand, but to have independence of Heroku for local testing, we will use thedotenvlibrary which is not much different.

This library pushes a set of environment variables when the server starts from a.envfile if found. We will have a .env file only in the local environment but we will not push it to Heroku as stated in the .gitignore above. Heroku instead uses configuration variables to feed the same information.

 

 

First, let us install the dotenv library

$npm install dotenv --save

Then, let us add a .env file to the root of the project.env

DBX_APP_KEY='<appkey_in_dropbox_console>'
DBX_APP_SECRET='<appsecret_in_dropbox_console>'
OAUTH_REDIRECT_URL='http://localhost:3000/oauthredirect'
SESSION_ID_SECRET='cAt2-D0g-cAW'

Finally, replace the whole config file to the code below. Notice how we are reading several variables from the environment (which is loaded at startup from the .env file)config.js

require('dotenv').config({silent: true});

module.exports = {
        DBX_API_DOMAIN: 'https://api.dropboxapi.com',
        DBX_OAUTH_DOMAIN: 'https://www.dropbox.com',
        DBX_OAUTH_PATH: '/oauth2/authorize',
        DBX_TOKEN_PATH: '/oauth2/token',
        DBX_LIST_FOLDER_PATH:'/2/files/list_folder',
        DBX_LIST_FOLDER_CONTINUE_PATH:'/2/files/list_folder/continue',
        DBX_GET_TEMPORARY_LINK_PATH:'/2/files/get_temporary_link',
        DBX_APP_KEY:process.env.DBX_APP_KEY,
        DBX_APP_SECRET:process.env.DBX_APP_SECRET, 
        OAUTH_REDIRECT_URL:process.env.OAUTH_REDIRECT_URL,
        SESSION_ID_SECRET:process.env.SESSION_ID_SECRET,
}

Since those variables won’t exist on Heroku, we need to manually add them. So in your app in Heroku, click onSettingsand then onReveal config vars

Then manually add the variables with the proper values

👁️Notice something important here. We changed the OAUTH_REDIRECT_URL tohttps://dbximgs.herokuapp.com/oauthredirect. In this field you need to put the name of your app in the following way:

https://.herokuapp.com/oauthredirect

 

 

5. As we are now using a different redirect URL for the authentication, we need to also add it to theDropbox app console.

6. When you deploy your app to Heroku, it installs all the libraries and dependencies from your package.json file, but the problem we will see is that Node.JS itself might be a version not compatible yet with elements of ES7 we put in our code like the async/await calls. To avoid this, we need to set the Node.JS dependency in the package.json file. For this add the following lines right before thedependencies

package.json

"engines": {
  "node": "~8.2.1"
},

It will look something like thispackage.json

{
  "name": "dbximgs",
  "version": "0.0.0",
  "private": true,
  "scripts": {
    "start": "node ./bin/www"
  },
  "engines": {
    "node": "~8.2.1"
  },
  "dependencies": {
      //many libraries listed here
  }
}

7. We use Redis to store sessions and luckily there is a free Heroku addin that can be configured only with a few steps:

• First, go to theHeroku Redis addin page
• Click onInstall Heroku Redisbutton
• Select your application from the dropdown list
• Choose the free tier and pressprovision. If you don’t have a credit card on file this will fail, so you will need to add one to continue in the billing settings of your profile.

Now you will see the addin in your app

There is one more step you need to change in your code for Redis to work. You need to add an environment variable when you create the database client in the app.js file. When this runs locally, this value will be empty, but when Heroku calls it, it will add a variable it has added to our config vars when you deployed the plugin.

app.js

//var client = redis.createClient(); 
var client = redis.createClient(process.env.REDIS_URL);

You can see it yourself in the settings page of the Heroku app if you click on Reveal config vars

 

8.Seems we have all the elements in place to push the magic button. (make sure you are logged to Heroku in your console, otherwise run theheroku logincommand at the root level of your project.

now run

 

$git add --all git commit -m "Heroku ready code"

$git push heroku master

$git add --all git commit -m "Heroku ready code" 
$git push heroku master

 

which will start the deploy and will show you something like this

You can also check the Heroku logs to make sure server is running correctly

$heroku logs --tail

Something like this means things are working fine

9. You can now go and test your app!!🤡

 

If you are not sure of the link, you can start it from the Heroku console using the Open App button. Or run theheroku opencommand in the console.

It will be something like this

https://.herokuapp.com

👁️Remember to add some images to the folder if the Dropbox account you are linking is new. Otherwise, you will simply see a sad white page.🎯The source code at this point can be found inthis link

8. Security considerations and actions

In general, there are a set of security measures we can take to protect our app. I am checking the ones we have already implemented.

Oauth

• 💚OAuth code flow where token is not exposed to the Web browser
• 💚Check state parameter on the OAuth flow to avoid CSRF
• 💚Store token on a fresh session (regenerate the session)

Cookies

• 💚Cookies: disabling scripts to read/write cookies from browser in the session configuration. By default the HttpOnly attribute is set.
• 💚Cookies: enforcing same domain origin (default on the session configuration)
• 🔴Cookies: make sure cookies are transported only via https.😱we will fix it later.

Securing headers

• Good information inthis blog post, but here is the summary of what we should care about.
• 🔴Strict-Transport-Securityenforces secure (HTTP over SSL/TLS) connections to the server
• 🔴X-Frame-Optionsprotection against clickjacking or disallowing to be iframed on another site.
• 🔴X-XSS-ProtectionCross-site scripting (XSS) filter
• 🔴X-Content-Type-Optionsprevents browsers from MIME-sniffing a response away from the declared content-type
• 🔴Content-Security-Policyprevents a wide range of attacks, including Cross-site scripting and other cross-site injections

The blogpost above has more security considerations if you intend to go deeper on the topic.

To secure the headers above, we will use thehelmetlibrary.
💡This is agood blog post to read about helmet

$npm install helmet --save

And add the following code to the app.js file to set the headers. Notice that we are only allowing scripts from the ajax.googleapi (this is where we find the jQuery library). Another option is to simply copy the file locally, for that, change the reference in the page.js file.

app.js

var helmet = require('helmet'); //Headers security!! app.use(helmet()); // Implement CSP with Helmet app.use(helmet.contentSecurityPolicy({ directives: { defaultSrc: ["'self'"], scriptSrc: ["'self'","https://ajax.googleapis.com/"], styleSrc: ["'self'"], imgSrc: ["'self'","https://dl.dropboxusercontent.com"], mediaSrc: ["'none'"], frameSrc: ["'none'"] }, // Set to true if you want to blindly set all headers: Content-Security-Policy, // X-WebKit-CSP, and X-Content-Security-Policy. setAllHeaders: true }));

With this we have secured the headers now🤠

• 💚Securing headers

Let us know fix the cookie transport issue. The best thing to do here is to enable http for development purposes and only allow https for production. Development and production can be set with the NODE_ENV env variable. Heroku is by defauld set to production, the local host is development by default. You can modify this behaviorfollowing these steps

After the sess variable is initialized (before the app.use) in the apps.js add the following codeapp.js

//cookie security for production: only via https if (app.get('env') === 'production') { app.set('trust proxy', 1) // trust first proxy sess.cookie.secure = true // serve secure cookies }

It is important to do thetrust the first proxyfor Heroku as any requests enters via https to Heroku but the direct internal call to our middleware is http via some load balancer.

And we are done!👊

• 💚Cookies: make sure cookies are transported only via https

Now you want to push this to heroku

git add --all git commit -m "security improvements" git push heroku master

🎯The source code at this point can be found inthis link

Checking dependency vulnerabilities

The great thing about Node.JS is that you usually find a library that does exactly what you want. But it comes to a great cost, libraries get outdated and then you find yourself in🔥library hell🔥when they have specific vulnerabilities that get patched. Think about it, you have dependencies that have dependencies and suddenly you have hundreds of dependencies and libraries that might be outdated and vulnerable.

A good way to check you are protected against vulnerabilities to specific dependencies in your code is thensp checkcommand. It will tell you which libraries have been compromised, what are the patched versions, where to find them and the version you have.

so run

$nsp check

and you will get a bunch of tables that look like this

As with Heroku we don’t actually push thenode_modulespackage, it makes sense to patch those files that are directly stated in thepackage.jsonfile by simply changing the version required. But make sure you test in case there were major changes for that dependency. Every time you push your code to Heroku, it runs thenpm installcommand recreating the node_modules folder.

If the vulnerability is in a library within one of your projects dependencies, check if updating the dependency will fix the issue. Otherwise you have three options: accept the risk of keeping it, replace the library for a similar without the vulnerability or finally, patch it yourself and then actually push all the libraries to Heroku yourself.

9. Production and things to do from here

The master branch of this tutorial also includes a method to logout which revokes the token and deletes the session, it also removes some unused files.check it out.

This is an additional list of things to do to make this tutorial fully production material.

1. Manage the errors: for this tutorial, all the errors will be propagated to the app.js page that will set the right error code on the http response and display the error template. res.render('error');

You could make that error page look nicer.

1. The front end was barely worked, you can make it look much nicer.
2. Add app icons to the Dropbox page so the authorization looks better.
3. Probably the most important one will be pagination on the images. You can call Dropbox specifying a number of items you want back. Then Dropbox will give you a cursor to iterate. Adding that will just make this tutorial much longer, so that’s a good task for you if you want to extend it.